Bag Of Visual Words Research Articles

Unsupervised Learning for Concept Detection in Medical Images: A Comparative Analysis

As digital medical imaging becomes more prevalent and archives increase in size, representation learning exposes an interesting opportunity for enhanced medical decision support systems. On the other hand, medical imaging data is often scarce and short on annotations. In this paper, we present an assessment of unsupervised feature learning approaches for images in biomedical literature which can be applied to automatic biomedical concept detection. Six unsupervised representation learning methods were built, including traditional bags of visual words, autoencoders, and generative adversarial networks. Each model was trained, and their respective feature spaces evaluated using images from the ImageCLEF 2017 concept detection task. The highest mean F1 score of 0.108 was obtained using representations from an adversarial autoencoder, which increased to 0.111 when combined with the representations from the sparse denoising autoencoder. We conclude that it is possible to obtain more powerful representations with modern deep learning approaches than with previously popular computer vision methods. The possibility of semi-supervised learning as well as its use in medical information retrieval problems are the next steps to be strongly considered.

Transportation Object Detection with Bag of Visual Words Model by PLSA and MLP

Visual big data is an essential and significant research topic, due to its diverse applications. In this paper, a new visual detection method for transportation is proposed based on probabilistic latent semantic analysis with visual data. We detect the distinctiveness by integrating three steps as follows: first, representing the co-ocurrence matrix of images, which were vectorized using the bag of visual words (BoVW) framework; then calculating the histograms of the visual words of each class; and finally applying the test images as the visual words. A multilayer perceptron (MLP) is used as the classification method in our system. The visual words are extracted by sampling the patches from the current image. A new topology of the neural network for the BoVW model is proposed, and management of the learning rate by reducing at specific iterations is exploited. The Probabilistic latent semantic analysis (PLSA) is compared to the MLP using the Caltech 256 datasets. The classes used include cars, motorbikes, and horses. The results of the experiment show that the MLP outperforms current methods in predicting transportation objects, and properly approximates the transportation detection function with extracted local features. It shows that the proposed method yields about 4.4% higher accuracy than the conventional PLSA for all classes.

Boosting image classification through semantic attention filtering strategies

Saliency Maps, frequently used to highlight significant information, can be combined with other paradigms, such as Bag of Visual Words (BoVW), to improve image description when the saliency regions correspond closely with the objects of interest. In this paper, we present three attention filtering strategies based on their saliency map that improve image classification using the BoVW framework, Spatial Pyramid Matching (SPM) and Convolutional Neural Networks (CNN) features. Firstly, we demonstrate how the blurring factor used in the Hou’s image signature algorithm determines what information remains and impacts to the obtained accuracy in image classification. Next, we propose AutoBlur, a simple but effective approach to automatically select this factor. Then, based on AutoBlur, we introduce two variants of our approach SARF (Semantic Attention Region Filtering), to semantically remove non-relevant regions through a Mean Shift segmentation. The first one is based on the intersection of the Hou’s image attention areas with its Mean Shift segmentation, while the second one discards regions using a key point voting system that relies on the Euclidean distance. The experiments carried out showed that the methods of Semantic Attention Filtering that we are proposing could be successfully used with both BoVW, SPM and CNN’s in most of the evaluated situations. In the five datasets assessed, all the three proposed methods outperform the baseline when using BoVWs in almost every case. For Spatial Pyramid Matching, the behaviour is similar, finding that the baseline is superior to our proposals in only one of the datasets used. In the case of CNN’s, our filtering proposal outperforms the baseline in two datasets, being very similar to it in the other cases.

Hybrid Classification in Bag of Visual Words Model

Classification of images by means of the BOVW method is well known and applied in different recognition projects, this method rely on three phases: detection and extraction of characteristics, representation of the image and finally the classification. SIFT, Kmeans and SVM is the most accepted combination. This article aims to demonstrate that this combination is not always the best choice for all types of datasets, different training sets of images were created from scratch and will be used for the bag of visual words model: the first phase of detection and extraction, SIFT will be used, later in the second phase a dictionary of words will be created through a clustering process using K-means, EM, K-means in combination with EM, finally, for classification it will be compared the algorithms of SVM, Gaussian NB, KNN, Decision Tree, Random Forest, Neural Network and AdaBoost in order to determine the performance and accuracy of every method.

Content and context based image retrieval classification based on firefly-neural network

In this study presents an improved classification and feature reduction techniques are newly introduced in our proposed search area of image retrieval. Especially we are including a shape feature that has been so active and successful in the past few years. In our proposed approach the texture feature is extracted by using improved multi texton technique and GLCM technique and the textual features are keywords, annotations. The Visual features are extracted with the aid of a bag of visual words (BOW) also it can be approved by employ Scaling Invariant Feature Transform (SIFT) and Fuzzy C-Means Clustering technique the features are edge detection, corner detection. Improving a performance and accuracy newly we are including a shape based features the features are Active Appearance Model (AAM) and shape surface plot (SURF). In the second phase, we are including feature reduction method is applied to reduce the features’ space without losing the accuracy of classification in this reduction phases we are including an OLPP technique to perform a feature reduction phases. After that, the images are classified with the aid of modified neural network algorithm. This will classify the images in the different class in order to improve the precision and recall rate. After classification, we are including a squared Euclidian distance method to retrieve minimum distance image the similar images are retrieved from the relevant class as per the given query image.

Scene search based on the adapted triangular regions and soft clustering to improve the effectiveness of the visual-bag-of-words model

The storage size of the image and video repositories are growing day by day due to the extensive use of digital image acquisition devices. The position of an object within an image is obtained by analyzing the content-based properties like shape, texture, and color, while compositional properties present the image layout and include the photographic rule of composition. The high-quality images are captured on the basis of the rule of thirds that divide each image into nine square areas. According to this rule, salient objects of an image are placed on the intersection points or along the imagery lines of the grid to capture the position of the salient objects. To improve image retrieval performance, visual-bag-of-words (VBoW) framework-based image representation is widely used nowadays. According to this framework, the spatial relationship between salient objects of an image is lost due to the formation of a global histogram of the image. This article presents a novel adapted triangular area-based technique, which computes local intensity order pattern (LIOP) features, weighted soft codebooks, and triangular histograms from the four triangular areas of each image. The proposed technique adds the spatial contents from four adapted triangular areas of each image to the inverted index of the VBoW framework, solve overfitting problem of the larger sizes of the codebook, and overwhelmed the problem of the semantic gap. The experimental results and statistical analysis performed on five image collections show an encouraging robustness of the proposed technique that is compared with the recent CBIR techniques.

Image classification by addition of spatial information based on histograms of orthogonal vectors.

The Bag-of-Visual-Words (BoVW) model is widely used for image classification, object recognition and image retrieval problems. In BoVW model, the local features are quantized and 2-D image space is represented in the form of order-less histogram of visual words. The image classification performance suffers due to the order-less representation of image. This paper presents a novel image representation that incorporates the spatial information to the inverted index of BoVW model. The spatial information is added by calculating the global relative spatial orientation of visual words in a rotation invariant manner. For this, we computed the geometric relationship between triplets of identical visual words by calculating an orthogonal vector relative to each point in the triplets of identical visual words. The histogram of visual words is calculated on the basis of the magnitude of these orthogonal vectors. This calculation provides the unique information regarding the relative position of visual words when they are collinear. The proposed image representation is evaluated by using four standard image benchmarks. The experimental results and quantitative comparisons demonstrate that the proposed image representation outperforms the existing state-of-the-art in terms of classification accuracy.

Dynamic Match Kernel With Deep Convolutional Features for Image Retrieval.

For image retrieval methods based on bag of visual words, much attention has been paid to enhancing the discriminative powers of the local features. Although retrieved images are usually similar to a query in minutiae, they may be significantly different from a semantic perspective, which can be effectively distinguished by convolutional neural networks (CNN). Such images should not be considered as relevant pairs. To tackle this problem, we propose to construct a dynamic match kernel by adaptively calculating the matching thresholds between query and candidate images based on the pairwise distance among deep CNN features. In contrast to the typical static match kernel which is independent to the global appearance of retrieved images, the dynamic one leverages the semantical similarity as a constraint for determining the matches. Accordingly, we propose a semantic-constrained retrieval framework by incorporating the dynamic match kernel, which focuses on matched patches between relevant images and filters out the ones for irrelevant pairs. Furthermore, we demonstrate that the proposed kernel complements recent methods, such as hamming embedding, multiple assignment, local descriptors aggregation, and graph-based re-ranking, while it outperforms the static one under various settings on off-the-shelf evaluation metrics. We also propose to evaluate the matched patches both quantitatively and qualitatively. Extensive experiments on five benchmark data sets and large-scale distractors validate the merits of the proposed method against the state-of-the-art methods for image retrieval.

A proposed method for the improvement in biometric facial image recognition using document-based classification

This paper mainly focuses on improving the recognition rate and reducing the recognition time in facial image recognition application. The existing methods are based on statistical or neural network or fuzzy-based feature extraction. In this study, the feature extraction followed by classification method is carried out based on documentation-based approach called bag of visual words (BOVW). In BOVW method, the feature vectors were extracted on the basis of scale invariant feature transform (SIFT) and classified by support vector machine (SVM). In train 50% and test 50% strategy, four standard face databases were tested with BOVW documentation approach. For the face databases such as Our Databases of Face Research Lab (ORL), Surveillance, Yale, Face recognition technology (FERET), this method produced 98, 82, 89.33, and 97.9798% of recognition rate, respectively. In the leave-one-out strategy, nine standard face databases were tested. The BOVW method gave 100% of recognition rate for face databases such as Cohn–Kanade (CK+), Georgia Tech, Morphological, Surveillance, Yale and YaleB, whereas it gave 99.772% of recognition rate for ORL and 97.9798% for FERET face databases. Our choice of BOVW + SVM is a better approach to increase classification rate and also reduce recognition time.

From BoVW to VLAD with KAZE features: Offline signature verification considering cognitive processes of forensic experts

The widespread use of handwritten signatures for identity authentication has resulted in a need for automated verification systems. However, there is room for improvement in the performance of automated systems compared to human analysis, particularly that of forensic document examiners (FDEs), under a wide range of conditions. As any inaccuracy in analysis can cause serious problems, further research is required to improve performance. In this study, to improve the performance of offline signature verification under various writing conditions, new approaches using feature encoding methods of a bag-of-visual words (BoVW) and a vector of locally aggregated descriptors (VLAD) are adopted. The proposed method considers current knowledge about the cognitive processes used by FDEs to improve performance. The proposed method incorporates an approach based on BoVW and VLAD with local features to mimic the cognitive processes of FDEs for feature extraction. In addition, KAZE features detected in strokes and background space are employed as local features to enhance discriminative power. The performance of the proposed approach is demonstrated using the publicly-available CEDAR and MCYT-75 signature datasets.

Sign text detection in street view images using an integrated feature

Based on Bag of Visual Words (BoVWs) model, this paper proposes a novel method using an integrated feature to detect sign text in the street view images. BRISK features are first extracted from the street view images for dictionary learning. The Self-Growing and Self-Organized Neural Gas (SGONG) network is then used to cluster adaptively the extracted BRISK descriptors for generating visual words. The histogram of visual words is further calculated to form the appearance feature of the sign text. For eliminating the color differences and further highlighting the histogram similarity of all colors of signs, a color invariant histogram, called CIHS histogram, is presented to represent the color information of the sign text. By integrating the visual words histograms and CIHS histograms, an integrated descriptor, called Appearance and Color (A&C) descriptor, is specifically designed as the input features for cascade-Adaboost classifier. In the multi-scale sliding window text sign detection, integral image is applied to the spatial distribution map of each visual word for avoiding repeated extraction of features. Experimental results demonstrate that the proposed method outperforms the state-of-the-art methods and the detectors with the traditional descriptors.

Color-Boosted Saliency-Guided Rotation Invariant Bag of Visual Words Representation with Parameter Transfer for Cross-Domain Scene-Level Classification

Scene classification on remote sensing imagery is usually based on supervised learning but collecting labelled data in remote sensing domains is expensive and time-consuming. Bag of Visual Words (BOVW) achieves great success in scene classification but there exist problems in domain adaptation tasks, such as the influence of background and the rotation transformation on BOVW representation, and the transfer of SVM parameters from the source domain to the target domain, which may lead to decreased cross-domain scene classification performance. In order to solve the three problems, Color-boosted saliency-guided rotation invariant bag of visual words representation with parameter transfer is proposed for cross-domain scene classification. The global contrast-based salient region detection method is combined with the color-boosted method to increase the accuracy of detected salient regions and reduce the effect of background information on the BOVW representation. The rotation invariant BOVW representation is also proposed by sorting the BOVW representation in each patch in order to decrease the effect of rotation transformation. The several best configurations in the source domain are also applied to the target domain so as to reduce the distribution bias between scenes in the source and target domain. These configurations deliver the top classification performance the optimal parameter in the target domain. The experimental results on two benchmark datasets confirm that the proposed method outperforms most previous methods in scene classification when instances in the target domain are limited. It is also proved that color boosted global contrast-based salient region detection (CBGCSRD) method, rotation invariant BOVW representation, and transfer of SVM parameters from the source to the target domain are all effective in improving the classification accuracy with 2.5%, 3.3%, and 3.1%. These three contributions may increase about 7.5% classification accuracy in total.

A Deep-Local-Global Feature Fusion Framework for High Spatial Resolution Imagery Scene Classification

High spatial resolution (HSR) imagery scene classification has recently attracted increased attention. The bag-of-visual-words (BoVW) model is an effective method for scene classification. However, it can only extract handcrafted features, and it disregards the spatial layout information, whereas deep learning can automatically mine the intrinsic features as well as preserve the spatial location, but it may lose the characteristic information of the HSR images. Although previous methods based on the combination of BoVW and deep learning have achieved comparatively high classification accuracies, they have not explored the combination of handcrafted and deep features, and they just used the BoVW model as a feature coding method to encode the deep features. This means that the intrinsic characteristics of these models were not combined in the previous works. In this paper, to discover more discriminative semantics for HSR imagery, the deep-local-global feature fusion (DLGFF) framework is proposed for HSR imagery scene classification. Differing from the conventional scene classification methods, which utilize only handcrafted features or deep features, DLGFF establishes a framework integrating multi-level semantics from the global texture feature–based method, the BoVW model, and a pre-trained convolutional neural network (CNN). In DLGFF, two different approaches are proposed, i.e., the local and global features fused with the pooling-stretched convolutional features (LGCF) and the local and global features fused with the fully connected features (LGFF), to exploit the multi-level semantics for complex scenes. The experimental results obtained with three HSR image classification datasets confirm the effectiveness of the proposed DLGFF framework. Compared with the published results of the previous scene classification methods, the classification accuracies of the DLGFF framework on the 21-class UC Merced dataset and 12-class Google dataset of SIRI-WHU can reach 99.76%, which is superior to the current state-of-the-art methods. The classification accuracy of the DLGFF framework on the 45-class NWPU-RESISC45 dataset, 96.37 ± 0.05%, is an increase of about 6% when compared with the current state-of-the-art methods. This indicates that the fusion of the global low-level feature, the local mid-level feature, and the deep high-level feature can provide a representative description for HSR imagery.

Visual Distinctiveness Detection of Pedestrian based on Statistically Weighting PLSA for Intelligent Systems

Intelligent Systems for autonomous vehicles including drone, robot vision, and video surveillance, need to distinguish pedestrian from other object. Pedestrian detection is an essential and significant research topic due to its diverse applications. In this paper, a new visual distinctiveness detection method for pedestrian is proposed based on the statistically weighting probabilistic latent semantic analysis. We detect the distinctiveness by integrating three steps as follows: first representing the co-ocurrence matrix of images, which were vectorized using the bag of visual words (BoVW) framework; then calculating the weights through the histograms of visual words of each class; and finally applying the weights to the test images as the distinctiveness of visual words. The probabilistic latent semantic analysis (PLSA) was used as classification method in our system. We extracted the weighted visual words by sampling the patches from the current image. The proposed method was compared to the PLSA using the Caltech 256 datasets. The classes used include pedestrians, cars, motorbikes, airplanes and horses. The results of the experiment show that the proposed method outperforms current methods in predicting pedestrians and transportation objects.

시각 단어 가방 기반의 가로수 종류 및 상태 분류 기법

가로수는 도시의 생활환경을 개선하는데 큰 역할을 하고 있으며 그 중요성이 점점 강조되어 매년 가로수의 개체 수는 늘어가고 있다. 최근 무선 주파수 식별 기반의 가로수 관리 시스템, 컴퓨터 비전 기반의 잎사귀 분류 기법, 또는 초분광 영상 기반의 열대우림 수종 식별 기법과 같은 유사내용을 다룬 연구가 소개되고 있지만, 구축비용이 많이 들거나 잎사귀 부분만 인식하기 때문에 도로가에서 촬영된 영상에서 가로수종이나 훼손 상태를 구별하고 질병까지 관리하기란 역부족이다. 따라서 이 논문에서는 도로가에 심겨진 가로수 영상으로부터 훼손 상태나 심지어 황화 현상과 같은 질병까지 검출할 수 있는 기법을 제안하고자 한다. 특히 국부 영역에서 에지의 특징을 추출해서 가로수종 및 상태에 관한 특징을 히스토그램으로 모델링한 시각 단어 가방 기법을 사용하였다. 분류기로는 서포트 벡터 머신을 사용하였고, 그 결과 약 78%의 정인식률을 달성할 수 있었다.

시각 단어 가방 기반의 가로수 종류 및 상태 분류 기법

가로수는 도시의 생활환경을 개선하는데 큰 역할을 하고 있으며 그 중요성이 점점 강조되어 매년 가로수의 개체 수는 늘어가고 있다. 최근 무선 주파수 식별 기반의 가로수 관리 시스템, 컴퓨터 비전 기반의 잎사귀 분류 기법, 또는 초분광 영상 기반의 열대우림 수종 식별 기법과 같은 유사내용을 다룬 연구가 소개되고 있지만, 구축비용이 많이 들거나 잎사귀 부분만 인식하기 때문에 도로가에서 촬영된 영상에서 가로수종이나 훼손 상태를 구별하고 질병까지 관리하기란 역부족이다. 따라서 이 논문에서는 도로가에 심겨진 가로수 영상으로부터 훼손 상태나 심지어 황화 현상과 같은 질병까지 검출할 수 있는 기법을 제안하고자 한다. 특히 국부 영역에서 에지의 특징을 추출해서 가로수종 및 상태에 관한 특징을 히스토그램으로 모델링한 시각 단어 가방 기법을 사용하였다. 분류기로는 서포트 벡터 머신을 사용하였고, 그 결과 약 78%의 정인식률을 달성할 수 있었다.

Determining Anxiety in Obsessive Compulsive Disorder through Behavioural Clustering and Variations in Repetition Intensity

Background and objectivesOver the last decade, the application of computer vision techniques to the analysis of behavioural patterns has seen a considerable increase in research interest. One such interesting and recent application is the visual behavioural analysis of mental disorders. Despite the very recent surge in interest in this area, relatively little has been done thus far to assist individuals living with Obsessive Compulsive Disorder. The work proposed herein represents a proof of concept system designed to demonstrate the efficacy of such an approach, from the computational perspective. The specific focus of this work lies in demonstrating a mechanism for clustering different kinds of Obsessive Compulsive Disorder behaviours and then comparing new behaviours to existing behaviours to determine the approximate level of anxiety represented by a compulsive behaviour. MethodsThe proposed system uses Temporal Motion Heat Maps, SURF descriptors, a visual bag of words model and SVM-based classification to categorise representations of various behaviours commonly seen in OCD. Moreover, we apply a set of statistical measures to the images in a given category in order to derive an approximate anxiety level for a given compulsive behaviour. This proof of concept is an essential step in the direction of integrating computational approaches into the treatment of psychiatric conditions such as Obsessive Compulsive Disorder, for more effective recovery. ResultsResults gleaned from experimental simulations indicate that the proposed system is capable of correctly classifying different types of simulated Obsessive Compulsive Disorder behaviour classes 75% of the time, with the misclassifications almost exclusively occurring when two behavioural clusters appear highly similar. Based on this information the proposed system is then able to assign an approximate behavioural anxiety level to the compulsive behaviours that meets the approval of a mental health professional. ConclusionsThe proposed system demonstrates a good ability to categorise various types of simulated OCD behaviour, in addition to establishing an approximate anxiety level for a given compulsive behaviour. This research demonstrates strong potential for the use of such systems in assisting mental health professionals looking to better understand their patients’ condition and treatment progress across time.

A robust CBIR framework in between bags of visual words and phrases models for specific image datasets

One objective of the Content Based Image Retrieval research field is to propose new methodologies and tools to manage the increasing number of images available. Linked to a specific context of small expert datasets without prior knowledge, our research work focuses on improving the discriminative power of the image representation while keeping the same efficiency for retrieval. Based on the well-known bag of visual words model, we propose three different methodologies inspired by the visual phrase model effectiveness and a compression technique which ensures the same effectiveness for retrieval than the BoVW model. Our experimental results study the performance of our proposals on different well known benchmark datasets and show its good performance compared to other recent approaches.

Selection of relevant information to improve Image Classification using Bag of Visual Words

One of the main challenges in computer vision is image classification. Nowadays the number of images increases exponentially every day; therefore, it is important to classify them in a reliable way.The conventional image classification pipeline usually consists on extracting local image features, encoding them as a feature vector and classify them using a previously created model. With regards to feature codification, the Bag of Words model and its extensions, such as pyramid matching and weighted schemes, have achieved quite good results and have become the state of the art methods.The process as mentioned above is not perfect and computers, as well as humans, may make mistakes in any of the steps, causing a performance drop in classification. Some of the primary sources of error on large-scale image classification are the presence of multiple objects in the image, small or very thin objects, incorrect annotations or fine-grained recognition tasks among others.Based on those problems and the steps of a typical image classification pipeline, the motivation of this PhD thesis was to provide some guidelines to improve the quality of the extracted features to obtain better classification results. The contributions of the PhD thesis demonstrated how a good feature selection can contribute to improving the fine-grained classification, and that there would even be no need to have a big training data set to learn the key features of each class and to predict with good results.

Psoriasis image representation using patch-based dictionary learning for erythema severity scoring

Psoriasis is a chronic skin disease which can be life-threatening. Accurate severity scoring helps dermatologists to decide on the treatment. In this paper, we present a semi-supervised computer-aided system for automatic erythema severity scoring in psoriasis images. Firstly, the unsupervised stage includes a novel image representation method. We construct a dictionary, which is then used in the sparse representation for local feature extraction. To acquire the final image representation vector, an aggregation method is exploited over the local features. Secondly, the supervised phase is where various multi-class machine learning (ML) classifiers are trained for erythema severity scoring. Finally, we compare the proposed system with two popular unsupervised feature extractor methods, namely: bag of visual words model (BoVWs) and AlexNet pretrained model. Root mean square error (RMSE) and F1 score are used as performance measures for the learned dictionaries and the trained ML models, respectively. A psoriasis image set consisting of 676 images, is used in this study. Experimental results demonstrate that the use of the proposed procedure can provide a setup where erythema scoring is accurate and consistent. Also, it is revealed that dictionaries with large number of atoms and small patch sizes yield the best representative erythema severity features. Further, random forest (RF) outperforms other classifiers with F1 score 0.71, followed by support vector machine (SVM) and boosting with 0.66 and 0.64 scores, respectively. Furthermore, the conducted comparative studies confirm the effectiveness of the proposed approach with improvement of 9% and 12% over BoVWs and AlexNet based features, respectively.