Bag Of Visual Words Research Articles

Deep Learning Method for Recognition and Classification of Images from Video Recorders in Difficult Weather Conditions

The sustainable functioning of the transport system requires solving the problems of identifying and classifying road users in order to predict the likelihood of accidents and prevent abnormal or emergency situations. The emergence of unmanned vehicles on urban highways significantly increases the risks of such events. To improve road safety, intelligent transport systems, embedded computer vision systems, video surveillance systems, and photo radar systems are used. The main problem is the recognition and classification of objects and critical events in difficult weather conditions. For example, water drops, snow, dust, and dirt on camera lenses make images less accurate in object identification, license plate recognition, vehicle trajectory detection, etc. Part of the image is overlapped, distorted, or blurred. The article proposes a way to improve the accuracy of object identification by using the Canny operator to exclude the damaged areas of the image from consideration by capturing the clear parts of objects and ignoring the blurry ones. Only those parts of the image where this operator has detected the boundaries of the objects are subjected to further processing. To classify images by the remaining whole parts, we propose using a combined approach that includes the histogram-oriented gradient (HOG) method, a bag-of-visual-words (BoVW), and a back propagation neural network (BPNN). For the binary classification of the images of the damaged objects, this method showed a significant advantage over the classical method of convolutional neural networks (CNNs) (79 and 65% accuracies, respectively). The article also presents the results of a multiclass classification of the recognition objects on the basis of the damaged images, with an accuracy spread of 71 to 86%.

Quantitative analysis of visual codewords of a protein distance matrix.

3D protein structures can be analyzed using a distance matrix calculated as the pairwise distance between all Cα atoms in the protein model. Although researchers have efficiently used distance matrices to classify proteins and find homologous proteins, much less work has been done on quantitative analysis of distance matrix features. Therefore, the distance matrix was analyzed as gray scale image using KAZE feature extractor algorithm with Bag of Visual Words model. In this study, each protein was represented as a histogram of visual codewords. The analysis showed that a very small number of codewords (~1%) have a high relative frequency (> 0.25) and that the majority of codewords have a relative frequency around 0.05. We have also shown that there is a relationship between the frequency of codewords and the position of the features in a distance matrix. The codewords that are more frequent are located closer to the main diagonal. Less frequent codewords, on the other hand, are located in the corners of the distance matrix, far from the main diagonal. Moreover, the analysis showed a correlation between the number of unique codewords and the 3D repeats in the protein structure. The solenoid and tandem repeats proteins have a significantly lower number of unique codewords than the globular proteins. Finally, the codeword histograms and Support Vector Machine (SVM) classifier were used to classify solenoid and globular proteins. The result showed that the SVM classifier fed with codeword histograms correctly classified 352 out of 354 proteins.

NOVEL CBIR System Using Spark MAP-Reduce with a Firefly Macqueen’s K-Means Clustering Algorithm

Nowadays content-based image retrieval (CBIR) has been an active research area, demonstrating a feasible solution for recovering similar images from an image mine. The amount of digital images increases exponentially day by. Storage requirements for these images may increase from Gigabytes to Petabytes. Searching and retrieving the relevant images from such a large volume of image datasets, based on their contents, play a dynamic role in various applications of computer vision. The time taken to retrieve the images is more, and the accuracy of retrieved images is less in the existing systems. The limitation of Hadoop map-reduce is the lack of performing real-time tasks efficiently. A proficient content-based image retrieval framework based on Spark Map-Reduce with a Firefly MacQueen’s k-means clustering (FMKC) algorithm and Bag of visual word (BoVW) is proposed to achieve high accuracy for big data. The Apache spark programming can be used to productively recover pictures with less retrieval time and retrieve the accurate images from the big database that resembles the query image. The experimental results demonstrate that the method proposed in our work outperforms the state-of-the-art methods in terms of accuracy of the retrieved images and average retrieval time. The proposed system is 93% accurate, and it is easier to retrieve the images from the large database.

Swarm optimisation-based bag of visual words model for content-based X-ray scan retrieval

Classification and retrieval of medical images (MedIR) are emerging applications of computer vision for enabling intelligent medical diagnostics. Medical images are multi-dimensional and require specialised processing for the extraction of features from their manifold underlying content. Existing models often fail to consider the inherent characteristics of data and have thus often fallen short when applied to medical images. In this paper, we present a MedIR approach based on the bag of visual words (BoVW) model for content-based medical image retrieval. When it comes to any medical approach models, an imbalance in the dataset is one of the issues. Hence the perspective is also considering a balanced set of categories from an imbalanced dataset. The proposed work on BoVW model extracts features from each image are used to train supervised machine learning classifier for X-ray medical image classification and retrieval. During the experimental validation, the proposed model performed well with the classification accuracy of 89.73% and a good retrieval result using our filter-based approach.

Swarm optimisation-based bag of visual words model for content-based X-ray scan retrieval

Swarm optimisation-based bag of visual words model for content-based X-ray scan retrieval

Evaluation of Vehicle Camouflage Effectiveness Under a Complex Background Based on the Time-Limited Search Model

To comprehensively evaluate the camouflage effect of a camouflaged vehicle in a complex background based on the detection and identification processes of the time-limited search model, the visual saliency degree and feature similarity degree were used to represent the camouflage effect, and a camouflage effect evaluation method was proposed. For the visual saliency degree algorithm, in which the concepts of local saliency degree and target saliency degree were introduced, the visual saliency features of the camouflaged vehicle and background image were manually extracted, multiscale saliency matrices were constructed, and the comprehensive saliency function was used to calculate the visual saliency degree. Based on the bag of visual words model, the feature similarity degree algorithm extracted the local features from public images of the camouflaged vehicles, clustered these features to form a visual word bag, and calculated the feature similarity degree by comparing the local features of the camouflaged vehicle with the visual word bag. Through experimental verification and analysis of the results, the method considering complex landform background characteristics, battlefield target reconnaissance processes, and different camouflage statuses of the vehicle could objectively obtain the camouflage effect value of vehicles, quantitatively determine the advantageous application environment of different camouflaged states, and provide data support for camouflaged state design and concealed area selection.

Earth Observation Semantic Data Mining: Latent Dirichlet Allocation-Based Approach

Recent advances in remote sensing technology have provided (very) high spatial resolution Earth Observation data with abundant latent semantic information. Conventional data processing algorithms are not capable of extracting the latent semantic information form these data and harness their full potential. As a result, semantic information discovery methods, based on data mining techniques, such as latent Dirichlet allocation and bag of visual words models, can discover the latent information. Despite their crucial rule, there are only a few studies in the field of semantic data mining for remote sensing applications. This article is focused on this shortage. Three different scenarios are used to evaluate the semantic information discovery in various remote sensing applications, including both optical and synthetic aperture radar (SAR) data with different spatial resolutions. In the first scenario, semantic discovery method correlated the semantic perception of the user and machine to correct and enhance the user defined Ground Truth map in very high-resolution RGB data. The potential of the semantic discovery is evaluated for wildfire affected area detection in Sentinel-2 data in the second scenario. Finally, in the third scenario, the semantic discovery method is utilized to detect the misclassifications as well as the patches with ambiguous or multiple semantic labels in a Sentinel-1 SAR patch-based benchmark dataset to enhance the robustness and accuracy of the annotation in the dataset. Our results in these three scenarios demonstrated the capability of the data-mining-based semantic information discovery methods for various remote sensing.

Enhancing Scalability of Image Retrieval Using Visual Fusion of Feature Descriptors

Content-Based Image Retrieval (CBIR) is an approach of retrieving similar images from a large image database. Recently CBIR poses new challenges in semantic categorization of the images. Different feature extraction technique have been proposed to overcome the semantic breach problems, however these methods suffer from several shortcomings. This paper contributes an image retrieval system to extract the local features based on the fusion of scale-invariant feature transform (SIFT) and KAZE. The strength of local feature descriptor SIFT complements global feature descriptor KAZE. SIFT concentrates on the complete region of an image using high fine points of features and KAZE ponders on details of a boundary. The fusion of local feature descriptor and global feature descriptor boost the retrieval of images having diverse semantic classification and also helps in achieving the better results in large scale retrieval. To enhance the scalability of image retrieval bag of visual words (BoVW) is mainly used. The fusion of local and global feature representations are selected for image retrieval for the reason that SIFT effectively captures shape and texture and robust towards the change in scale and rotation, while KAZE have strong response towards boundary and changes in illumination. Experiments conducted on two image collections, namely, Caltech-256 and Corel 10k demonstrate the proposed scheme appreciably enhanced the performance of the CBIR compared to state-of-the-art image retrieval techniques.

Identification and Classification of Mechanical Damage During Continuous Harvesting of Root Crops Using Computer Vision Methods

Detecting sugar beetroot crops with mechanical damage using machine learning methods is necessary for fine-tuning beet harvester units. The Agrifac HEXX TRAXX harvester with an installed computer vision system was investigated. A video camera (24 fps) was installed above the turbine, which receives the dug-out beets after the digger and is connected to a single-board computer. At the preprocessing stage, static and insignificant image details were revealed. Canny edge detector and excess green minus excess red (ExGR) method were used. The identified areas were excluded from the image. The remaining areas were glued with similar areas of another image. As a result, the number of images entering the second stage of preprocessing was reduced by half. Then Otsu’s binarization was used. The main stage of image processing is divided into two sub-stages: detection and classification. The improved YOLOv4-tiny method was chosen for root crop detection using a single-board computer (SBC). This method allows processing up to 14 images of 416 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\times416$ </tex-math></inline-formula> pixels with 86% precision and 91% recall. To classify root crop damage, we considered two algorithms as candidates: 1. bag of visual words (BoVW) with a support vector machine (SVM) classifier using histogram of oriented gradients (HOG) and scale-invariant feature transform (SIFT) descriptors; 2. convolutional neural networks (CNN). Under normal lighting conditions, CNN showed the best accuracy, which was 99%. The implemented methods were used to detect and classify blurred images of sugar beetroots, which were previously rejected. For improved YOLOv4-tiny precision was 74% and recall was 70%. CNN classification accuracy was 92.6%.

A Framework for Interactive Visual Interpretation of Remote Sensing Data

Machine learning methods have shown tremendous success in understanding earth observation data; however, recently, there is a rising claim toward explainable machine learning approaches. Concerned researchers found interpretable visualizations to be greatly helpful in understanding how a model works. In this research, we propose a framework for interactive and interpretable visualization of remote sensing data using two machine learning models and an Elasticsearch (ES) database. Two explainable machine learning models, namely, bag-of-visual-words (BoVWs) and latent Dirichlet allocation (LDA) are chosen to model the data in an unsupervised manner and give a textual representation. The textualized remote sensing data are stored in an ES database. This framework offers several fast content-based search functionalities exploiting the full-text query capabilities of ES based on the respective representations and also offers an efficient storage mechanism for the data.

POLSAR Target Recognition Using a Feature Fusion Framework Based on Monogenic Signal and Complex-Valued Nonlocal Network

With the continuous development of synthetic aperture radar (SAR) systems, multi-polarization information has been increasingly applied to numerous fields, and automatic target recognition (ATR) in polarimetric SAR (POLSAR) has been recognized as vital problem. The SAR recognition methods can primarily fall into handcrafted feature-based algorithms and deep learning algorithms. The former exhibits excellent interpretability but insufficient generalization; the latter achieves stronger representational ability but relies on a considerable number of samples. To solve above problems, a feature fusion framework is proposed in this paper based on monogenic signal and complex-valued non-local network (CVNLNet) for POLSAR target recognition. The proposed feature fusion framework effectively uses the complementarity of handcrafted features and deep features, while making up for the disadvantages of single feature-based methods. First, a Mono-BOVW model is proposed based on monogenic signal and bag-of-visual-words (BOVW) model to extract handcrafted features, which can more fully mine the information covered in POLSAR data in multi-scale space. Moreover, CVNLNet is built for deep feature extraction to use both the amplitude and phase covered in POLSAR data. Next, a kernel discrimination correlation analysis algorithm (KDCA) is proposed to jointly analyze and transform the two features, so as to remove redundant information while retaining effective and discriminative information. Experiments on the MSTAR dataset and the GOTCHA dataset show that the proposed framework has superior performance on single polarimetric and fully polarimetric datasets.

Automatic Algorithm for Fractal Plant Art Image Similarity Feature Generation

With the popularity of smart devices and the Internet, the volume of multimedia data is growing rapidly, and content-based image retrieval (CBIR) can search for similar images from large-scale images to realize the utilization of the data. For data owners, outsourcing the management and maintenance of image data to cloud service providers can effectively reduce costs, but there is a privacy leakage problem. In this paper, we focus on image feature extraction, index design, and image similarity recognition methods under a dual server model with content-based image security similarity recognition as the research topic, the work done such as proposing a BOVW (Bag of Visual Word) feature-based image security similarity recognition scheme. The scheme combines SIFT (scale-invariant feature transform) feature secure extraction and locally sensitive hashing algorithm to achieve secure extraction of BOVW features of images. To protect the BOVW features of images, an inverted index based on word frequency division is designed, the index is stored in chunks, and an image secure similarity recognition scheme based on CNN (convolutional neural networks) features is proposed. The scalable hash index based on dimensional division is designed based on the image CNN features secure extraction algorithm. The security and performance of the proposed scheme are theoretically analyzed and experimentally verified. Based on different image datasets, the impact of different parameters on the performance of the scheme is tested, and optimized parameters are given. The experimental results show that the proposed scheme in this paper can effectively improve the efficiency of analyzing the similarity of plant botanical art images compared to the existing schemes.

Induction motor condition monitoring using infrared thermography imaging and ensemble learning techniques

In this paper, a novel noncontact and nonintrusive framework experimental method is used for the monitoring and the diagnosis of a three phase’s induction motor faults based on an infrared thermography technique (IRT). The basic structure of this work begins with this applying IRT to obtain a thermograph of the considered machine. Then, bag-of-visual-word (BoVW) is used to extract the fault features with Speeded-Up Robust Features (SURF) detector and descriptor from the IRT images. Finally, various faults patterns in the induction motor are automatically identified using an ensemble learning called Extremely Randomized Tree (ERT). The proposed method effectiveness is evaluated based on the experimental IRT images, and the diagnosis results show its capacity and that it can be considered as a powerful diagnostic tool with a high classification accuracy and stability compared to other previously used methods.

Segment-based bag of visual words model for urban land cover mapping using polarimetric SAR data

Producing high-resolution urban land cover maps is essential for decision-making and urban management. In this regard, Synthetic Aperture Radar (SAR), especially Polarimetric SAR (PolSAR), has served as a valuable data source to fulfill this task. Direct conversion of low-level features into high-level Land Cover (LC) concept may reduce the final classification accuracy. Therefore, mid-level representation models, such as Bag of Visual Words (BOVW), were employed to resolve the existing semantic gap challenge. In this paper, a Segment-based BOVW (Seg-BOVW) model was developed for urban land cover classification using PolSAR data. To this end, two PolSAR data over San Francisco Bay (SFB) and Flevoland (FL) acquired by RADARSAT-2 were employed to comprehensively evaluate the Seg-BOVW model's performance. First, to exploit the full potential of PolSAR data, 169 low-level features in four categories: (1) original, (2) polarimetric, (3) texture, and (4) decomposition features were extracted. Afterward, a Multi-Objective Genetic Algorithm (MOGA) was implemented to investigate the importance of low-level features for urban land cover mapping. This step resulted in selecting 14 and 6 low-level features, as the high contributing features, for SFB and FL datasets, respectively. The Seg-BOVW model achieved significant overall accuracies of 96.02% and 98.82% for SFB and FL, respectively, indicating the high potential of the proposed method for urban land cover classification. Furthermore, a comparison with other well-known algorithms of Support Vector Machine (SVM), Artificial Neural Networks (ANN), and Convolutional Neural Networks (CNN) was made, suggesting the capability of the Seg-BOVW model to improve the urban land cover classification results. Finally, the Seg-BOVW model was tested with two other PolSAR datasets acquired with different sensors over SFB to examine its applicability with different datasets.

A machine learning‐based method for automatic diagnosis of ankle fracture using X‐ray images

AbstractDue to the complex mechanism of ankle injury, the clinical diagnosis of ankle fracture is extremely difficult. In order to simplify the fracture diagnosis process, this study proposes an automatic diagnosis model of ankle fractures. Firstly, an ankle fracture classification method suitable for machine learning was developed. By dividing six fracture regions, multiple types of fractures were clarified, and a corresponding dataset was created accordingly. Secondly, the random forest model was used to preprocess the X‐ray images and segment the fracture foreground part of the X‐ray images. Finally, the bag of visual words (BoVW) was used for feature extraction, and classifiers were constructed in different regions for classification. The area under the curve (AUC) of XGBoost was 0.92 ± 0.06. The performance of the XGBoost has proved to be better compared with the SVM when training on a small dataset in each region.

A Modified HSIFT Descriptor for Medical Image Classification of Anatomy Objects

Modeling low level features to high level semantics in medical imaging is an important aspect in filtering anatomy objects. Bag of Visual Words (BOVW) representations have been proven effective to model these low level features to mid level representations. Convolutional neural nets are learning systems that can automatically extract high-quality representations from raw images. However, their deployment in the medical field is still a bit challenging due to the lack of training data. In this paper, learned features that are obtained by training convolutional neural networks are compared with our proposed hand-crafted HSIFT features. The HSIFT feature is a symmetric fusion of a Harris corner detector and the Scale Invariance Transform process (SIFT) with BOVW representation. The SIFT process is enhanced as well as the classification technique by adopting bagging with a surrogate split method. Quantitative evaluation shows that our proposed hand-crafted HSIFT feature outperforms the learned features from convolutional neural networks in discriminating anatomy image classes.

Product Based Classification of Bulk Food Grains using Bag of Visual Words and Deep Features

Product Based Classification of Bulk Food Grains using Bag of Visual Words and Deep Features

Implementation of Visual Sentiment Analysis on Flickr Images

Visual sentiment analysis is the way to automatically recognize positive and negative emotions from images, videos, graphics and stickers. To estimate the polarity of the sentiment evoked by images in terms of positive or negative sentiment, most of the state of the art works exploit the text associated with a social post provided by the user. However, such textual data is typically noisy due to the subjectivity of the user, which usually includes text useful to maximize the diffusion of the social post. This System will extract three views: visual view, subjective text view and objective text view of Flickr images and will give sentiment polarity positive, negative or neutral based on the hypothesis table. Subjective text view gives sentiment polarity using VADER (Valence Aware Dictionary and sEntiment Reasoner) and objective text view gives sentiment polarity with three convolution neural network models. This system implements VGG-16, Inception-V3 and ResNet-50 convolution neural networks with pre pre-trained ImageNet dataset. The text extracted through these three convolution networks is given to VADER as input to find sentiment polarity. This system implements visual view using a bag of visual word model with BRISK (Binary Robust Invariant Scalable Key points) descriptor. System has a training dataset of 30000 positive, negative and neutral images. All the three views’ sentiment polarity is compared. The final sentiment polarity is calculated as positive if two or more views gives positive sentiment polarity, as negative if two or more views gives negative sentiment polarity and as neutral if two or more views gives neutral sentiment polarity. If all three views give unique polarity then the polarity of the objective text view is given as output sentiment polarity.

Motor Fault Diagnosis Using Image Visual Information and Bag of Words Model

In order to solve the tough issue that is difficult to extract the representative fault features of the hybrid vibration signals from the various industrial applications, a motor fault diagnosis method based on image visual information and bag of words model (BoW) are proposed in this paper. Based on the redundancy information of raw signals, the mapping which is between the original fault signal and the visual information is obtained by the methods of SDP (Symmetrized Dot Pattern) image and dense SIFT (Scale Invariant Feature Transform) features. The bag of visual word (BoVW) and pyramid histogram intersection kernel support vector machine (PHIK-SVM) are applying to complete the fault diagnosis and state recognition of related motors. The test results reveal that the diagnostic accuracy of four fault states could reach 99.50%, moreover, the diagnostic accuracy of different operating conditions under each fault could reach 91.83% as well. Compared with the traditional methods, the proposed method is able to obtain the detailed characteristics of various motors from the perspective of visual knowledge, furthermore, it is more robust and without complex data processing.

Development of an Optimal Algorithm for Detecting Damaged and Diseased Potato Tubers Moving along a Conveyor Belt Using Computer Vision Systems

The article discusses the problem of detecting sick or mechanically damaged potatoes using machine learning methods. We proposed an algorithm and developed a system for the rapid detection of damaged tubers. The system can be installed on a conveyor belt in a vegetable store, and it consists of a laptop computer and an action camera, synchronized with a flashlight system. The algorithm consists of two phases. The first phase uses the Viola-Jones algorithm, applied to the filtered action camera image, so it aims to detect separate potato tubers on the conveyor belt. The second phase is the application of a method that we choose based on video capturing conditions. To isolate potatoes infected with certain types of diseases (dry rot, for example), we use the Scale Invariant Feature Transform (SIFT)—Support Vector Machine (SVM) method. In case of inconsistent or weak lighting, the histogram of oriented gradients (HOG)—Bag-of-Visual-Words (BOVW)—neural network (BPNN) method is used. Otherwise, Otsu’s threshold binarization—a convolutional neural network (CNN) method is used. The first phase’s result depends on the conveyor’s speed, the density of tubers on the conveyor, and the accuracy of the video system. With the optimal setting, the result reaches 97%. The second phase’s outcome depends on the method and varies from 80% to 97%. When evaluating the performance of the system, it was found that it allows to detect and classify up to 100 tubers in one second, which significantly exceeds the performance of most similar systems.