Image Descriptors Research Articles

Computer multimedia aided design and hand-drawn effect analysis based on grid resource sharing cooperative algorithm

Hand-drawn is one of the few visual descriptors that can directly represent visual content, and has significant research in the area of computer vision. Aiming at the problem of sparse features in the realm of hand-drawn image retrieval, hand-drawn images, and the easy deformation of hand-drawn images, this paper proposes a feature extraction method of grid resource sharing collaborative algorithm, which can be obtained utilizing precisely extracted semantic characteristics from hand-drawn images through computer multimedia-aided design Efficient and accurate retrieval results. First, the fundamental framework for obtaining semantic features is algorithm; then the attention model mechanism is the grid resource sharing collaborative introduced in the process of supervised training, and the attention structure block is introduced after the convolutional neural network’s bottom layer. To locate effective semantic features, In order to accomplish high-precision retrieval, the attention structure block combines channel attention structure and spatial attention structure to build the attention structure block. The last feature descriptor is then created by combining various semantic feature levels. The proposed strategy is practical and efficient, as demonstrated by the experimental findings on the comparison database Flickr15k. In addition, in the task of hand-drawn image classification, the proposed attention mechanism greatly improves the classification accuracy.

CSPFormer: A cross-spatial pyramid transformer for visual place recognition

Recently, the Vision Transformer (ViT), which applied the Transformer structure to various visual detection tasks, has outperformed convolutional neural networks (CNNs). Nonetheless, due to the lack of scale representation ability of the Transformer, how to extract the local features of the scene to effectively form a global descriptor is still a challenging problem. In the paper, we propose a Cross-Spatial Pyramid Transformer (CSPFormer) to learn the discriminative global descriptors from multi-scale visual features for efficient visual place recognition. Specifically, we first develop a pyramid CNN module that can extract multi-scale visual feature representations. Then, the extracted feature representations of multi-scales are input to multiple connected spatial pyramid Transformer modules that adaptively learn the spatial relationship of the different scale descriptors, where the multiple self-attention is applied to learn a global descriptor from discriminative local descriptors. CNN pyramid features and Transformer multi-scale features are mutually weighted to perform cross-spatial feature representation. The multiple self-attention enhances the long-term dependencies of multi-scale visual descriptors and reduces the computational cost. To obtain the final place-matching result accurately, the cosine function is used to calculate the spatial similarity between the two scenes. Experimental results on public place datasets show that the proposed method achieves state-of-the-art on large-scale visual place recognition tasks. Our model has achieved 94.7%, 92.8%, 91.3%, and 95.7% average recall based on the top 1% candidate scenario on KITTI, Nordland, VPRICE, and EuRoc datasets, respectively.

Semisupervised Vector Quantization in Visual SLAM Using HGCN

We present a novel vector quantization (VQ) module for the two state-of-the-art long-range simultaneous localization and mapping (SLAM) algorithms. The VQ task in SLAM is generally performed using unsupervised methods. We provide an alternative approach trough embedding a semisupervised hyperbolic graph convolutional neural network (HGCN) in the VQ step of the SLAM processes. The SLAM platforms we have utilized for this purpose are fast appearance-based mapping (FABMAP) and oriented fast and rotated short (ORB), both of which rely on extracting the features of the captured images in their loop closure detection (LCD) module. For the first time, we have considered the space formed by these SURF features, robust image descriptors, as a graph, enabling us to apply an HGCN in the VQ section which results in an improved LCD performance. The HGCN vector quantizes the SURF feature space, leading to a bag-of-word (BoW) representation construction of the images. This representation is subsequently used to determine LCD accuracy and recall. Our approaches in this study are referred to as HGCN-FABMAP and HGCN-ORB. The main advantage of using HGCN in the LCD section is that it scales linearly when the features are accumulated. The benchmarking experiments show the superiority of our methods in terms of both trajectory generation accuracy in small-scale paths and LCD accuracy and recall for large-scale problems.

Unobtrusive Cognitive Assessment in Smart-Homes: Leveraging Visual Encoding and Synthetic Movement Traces Data Mining.

The ubiquity of sensors in smart-homes facilitates the support of independent living for older adults and enables cognitive assessment. Notably, there has been a growing interest in utilizing movement traces for identifying signs of cognitive impairment in recent years. In this study, we introduce an innovative approach to identify abnormal indoor movement patterns that may signal cognitive decline. This is achieved through the non-intrusive integration of smart-home sensors, including passive infrared sensors and sensors embedded in everyday objects. The methodology involves visualizing user locomotion traces and discerning interactions with objects on a floor plan representation of the smart-home, and employing different image descriptor features designed for image analysis tasks and synthetic minority oversampling techniques to enhance the methodology. This approach distinguishes itself by its flexibility in effortlessly incorporating additional features through sensor data. A comprehensive analysis, conducted with a substantial dataset obtained from a real smart-home, involving 99 seniors, including those with cognitive diseases, reveals the effectiveness of the proposed functional prototype of the system architecture. The results validate the system's efficacy in accurately discerning the cognitive status of seniors, achieving a macro-averaged F1-score of 72.22% for the two targeted categories: cognitively healthy and people with dementia. Furthermore, through experimental comparison, our system demonstrates superior performance compared with state-of-the-art methods.

Efficient and Accurate Image Classification via Spatial Pyramid Matching and SURF Sparse Coding

Sparsely coded signal space representations do well in feature quantization. Instead of using standard vector quantization, the suggested method uses selective sparse coding to assemble the most important features of the appearance descriptors of nearby image patches. Inadequate coding also enables adjacent max pooling on some spatial scales, which, unlike the setup of average pooling in a histogram, links interpretation with scale invariance. The acquired visual illustration is the key contribution of this research; it performs outperform with linear-SVM, improves the model training's, which in turn speeds up testing with improves accuracy. The efficacy of the method we have employed has been substantiated through a series of experiments conducted on diverse datasets. Since top-performing image classification systems heavily rely on nonlinear SPM in mean of vector quantization, the trustworthy recommended linear SPM greatly increases the use of larger sets of training data. The method given herein deduces that the sparse coding of SURF feature’s function hampered a more comprehensive local appearance descriptor for general-purpose image processing. Experiments and comparisons are conducted on standard datasets such as Caltech-101, FTVL, and Corel-1000, using state-of-the-art techniques and descriptors. When compared over several other image categories and descriptors, the method provided here comes out on top.

Acoustic and visual geometry descriptor for multi-modal emotion recognition fromvideos

<span>Recognizing human emotions simultaneously from multiple data modalities (e.g., face, and speech) has drawn significant research interest, and numerous research contributions have been investigated in the affective computing community. However, most methods concentrate less on facial alignment and keyframe selection for audio-visual input. Hence, this paper proposed a new audio-visual descriptor, mainly concentrating on describing the emotion through only a few frames. For this purpose, we proposed a new self-similarity distance matrix (SSDM), which computes the spatial, and temporal distances through landmark points on the facial image. The audio signal is described through an asset of composite features, including statistical features, spectral features, formant frequencies, and energies. A support vector machine (SVM) algorithm is employed to classify both models, and the final results are fused to predict the emotion. Surrey audio-visual expressed emotion (SAVEE) and Ryerson multimedia research lab (RML) datasets are utilized for experimental validation, and the proposed method has shown significant improvement from the state of art methods.</span>

A Training-Free, Lightweight Global Image Descriptor for Long-Term Visual Place Recognition Toward Autonomous Vehicles

A Training-Free, Lightweight Global Image Descriptor for Long-Term Visual Place Recognition Toward Autonomous Vehicles

Robust zero‐watermarking algorithm based on discrete wavelet transform and daisy descriptors for encrypted medical image

AbstractIn the intricate network environment, the secure transmission of medical images faces challenges such as information leakage and malicious tampering, significantly impacting the accuracy of disease diagnoses by medical professionals. To address this problem, the authors propose a robust feature watermarking algorithm for encrypted medical images based on multi‐stage discrete wavelet transform (DWT), Daisy descriptor, and discrete cosine transform (DCT). The algorithm initially encrypts the original medical image through DWT‐DCT and Logistic mapping. Subsequently, a 3‐stage DWT transformation is applied to the encrypted medical image, with the centre point of the LL3 sub‐band within its low‐frequency component serving as the sampling point. The Daisy descriptor matrix for this point is then computed. Finally, a DCT transformation is performed on the Daisy descriptor matrix, and the low‐frequency portion is processed using the perceptual hashing algorithm to generate a 32‐bit binary feature vector for the medical image. This scheme utilises cryptographic knowledge and zero‐watermarking technique to embed watermarks without modifying medical images and can extract the watermark from test images without the original image, which meets the basic requirements of medical image watermarking. The embedding and extraction of watermarks are accomplished in a mere 0.160 and 0.411s, respectively, with minimal computational overhead. Simulation results demonstrate the robustness of the algorithm against both conventional attacks and geometric attacks, with a notable performance in resisting rotation attacks.

A Partitioned CAM Architecture with FPGA Acceleration for Binary Descriptor Matching

An efficient architecture for image descriptor matching that uses a partitioned content-addressable memory (CAM)-based approach is proposed. CAM is frequently used in high-speed content-matching applications. However, due to its lack of functionality to support approximate matching, conventional CAM is not directly useful for image descriptor matching. Our modifications improve the CAM architecture to support approximate content matching for selecting image matches with local binary descriptors. Matches are based on Hamming distances computed for all possible pairs of binary descriptors extracted from two images. We demonstrate an FPGA-based implementation of our CAM-based descriptor-matching unit to illustrate the high matching speed of our design. The time complexity of our modified CAM method for binary descriptor matching is O(n). Our method performs binary descriptor matching at a rate of one descriptor per clock cycle at a frequency of 102 MHz. The resource utilization and timing metrics of several experiments are reported to demonstrate the efficacy and scalability of our design.

Discriminative Embedded Oriented Local Pattern (D-EOLP): a new feature based image descriptor

Discriminative Embedded Oriented Local Pattern (D-EOLP): a new feature based image descriptor

PointMCD: Boosting Deep Point Cloud Encoders Via Multi-View Cross-Modal Distillation for 3D Shape Recognition

As two fundamental representation modalities of 3D objects, 3D point clouds and multi-view 2D images record shape information from different domains of geometric structures and visual appearances. In the current deep learning era, remarkable progress in processing such two data modalities has been achieved through respectively customizing compatible 3D and 2D network architectures. However, unlike multi-view image-based 2D visual modeling paradigms, which have shown leading performance in several common 3D shape recognition benchmarks, point cloud-based 3D geometric modeling paradigms are still highly limited by insufficient learning capacity due to the difficulty of extracting discriminative features from irregular geometric signals. In this article, we explore the possibility of boosting deep 3D point cloud encoders by transferring visual knowledge extracted from deep 2D image encoders under a standard teacher-student distillation workflow. Generally, we propose PointMCD, a unified multi-view cross-modal distillation architecture, including a pretrained deep image encoder as the teacher and a deep point encoder as the student. To perform heterogeneous feature alignment between 2D visual and 3D geometric domains, we further investigate visibility-aware feature projection (VAFP), by which point-wise embeddings are reasonably aggregated into view-specific geometric descriptors. By pair-wisely aligning multi-view visual and geometric descriptors, we can obtain more powerful deep point encoders without exhausting and complicated network modification. Experiments on 3D shape classification, part segmentation, and unsupervised learning strongly validate the effectiveness of our method. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">The code and data will be publicly available at</i> <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/keeganhk/PointMCD</uri> .

Wear mechanisms and severity level classification in iron ore transfer chute linings by propagating regional labels coded as embedding deep learning vectors

Wear mechanism severity analysis constitutes one of the principal strategies for selecting more wear-resistant materials. A machine learning model was developed to characterize and recognize wear mechanisms and severity in worn surfaces of industrial equipment. Particularly, deep features were computed from a convolutional net (VGG16 and Resnet50), while wear mechanisms and severity classification were computed from a Random Forest (RF) classifier. The histogram of oriented gradient (HoG) was also implemented to compute an image descriptor and thereafter a classification was carried out with Random Forest classifier. From a set of images of worn surfaces of transfer chute linings, two different datasets were employed to search independently among five wear mechanisms and four levels of wear severity. The proposed approach was successful in recognizing and mapping wear mechanisms and determining their severity levels, as confirmed by the high values of metrics obtained using the deep features. A comparison was carried out between the mass loss performance of the materials tested in the field and their wear severity levels recognized by the machine learning model. A satisfactory correlation was identified between the materials with the lowest and highest wear, as determined by their measured mass losses from a field test, and the corresponding levels of wear severity on their worn surfaces obtained by the machine learning model. This result demonstrates the potential of using machine learning for the worn surface analysis to support industrial equipment wear analysis.

Data Augmentation for Genetic Programming-Driven Late Merging of HOG and Uniform LBP Features for Texture Classification

The categorization of texture images requires the identification and extraction of meaningful keypoints, a crucial step in ensuring the precise representation of textured images. The literature has introduced numerous descriptors in order to detect and capture both local and global texture characteristics. These descriptors vary in their effectiveness depending on the specific application. However, it is generally accepted that they complement each other by compensating for their strengths and weaknesses. Therefore, many approaches focused on combining different types of image descriptors generating robust features that are invariant to image transformations. These solutions mostly focused on one way to deal with information and faced more problems as they need human intervention and a large set of data. The acknowledged benefits of combining multiple types of descriptors are accompanied by challenges specially. These challenges arise due to differences in properties, such as locality and sparsity, as well as the heterogeneity exhibited by generated features. To address this issue comprehensively, we present an approach that benefits from genetic programming techniques to generate and combine two distinct texture classifiers. It incorporates histograms of oriented gradients and local binary patterns descriptors, which capture different textures. To effectively fuse the results of both classifiers, the proposed approach employs a late fusion process along with data augmentation methods while using a limited amount of training data. To evaluate the performance of the proposed approach in texture image classification tasks, we have conducted extensive experiments on six challenging datasets encompassing various variations. We have also investigated its performance in a cross dataset problem where the model has been trained on instances of a dataset before being tested on samples of another dataset. The obtained results clearly demonstrate that the proposed approach surpasses other relevant low-level approaches, as well as existing GP-based and CNN methods specifically designed for describing and classifying textures. Thanks to its ability to simultaneously leverage multiple descriptors, the suggested solution shows a high potential for real-world applications, particularly in handling various image changes with robustness.

Loop Closure Detection Based on Compressed ConvNet Features in Dynamic Environments

In dynamic environments, convolutional neural networks (CNNs) often produce image feature maps with significant redundancy due to external factors such as moving objects and occlusions. These feature maps are inadequate as precise image descriptors for similarity measurement, hindering loop closure detection. Addressing this issue, this paper proposes feature compression of convolutional neural network output. The approach is detailed as follows: (1) employing ResNet152 as the backbone feature-extraction network, a Siamese neural network is constructed to enhance the efficiency of feature extraction; (2) utilizing KL transformation to extract principal components from the backbone network’s output, thereby eliminating redundant information; (3) employing the compressed features as input for NetVLAD to construct a spatially informed feature descriptor for similarity measurement. Experimental results demonstrate that, on the New College dataset, the proposed improved method exhibits an approximately 9.98% enhancement in average accuracy compared to the original network. On the City Center dataset, there is an improvement of approximately 2.64%, with an overall increase of about 23.51% in time performance. These findings indicate that the enhanced ResNet152 performs better than the original network in environments with more moving objects and occlusions.

Depth as attention to learn image representations for visual localization, using monocular images

Image retrieval algorithms are widely used in visual localization tasks. In visual localization, we can benefit from retrieving the images depicting same landmark taken from a pose similar to the query. However, state-of-the-art image retrieval algorithms are optimized mainly for landmark retrieval, and do not take camera pose into account. To address this limitation, we propose novel Depth Attention Network (DeAttNet). DeAttNet leverages both visual and depth information in learning a global image representation. Depth varies for similar features captured from different camera poses. Based on this insight, we employ depth within an attention mechanism to discern and emphasize the salient regions. In our method, we utilize monocular depth estimation algorithms to render depth maps. Compared to RGB only image descriptors, significant improvements are obtained with the proposed method on Mapillary Street Level Sequences, Pittsburgh and Cambridge Landmark datasets.

A robust and adaptable high-precision method for matching flipped SAR images based on an oriented descriptor

ABSTRACT The increasing imaging capability of spaceborne Synthetic Aperture Radar (SAR) systems makes a challenging task in matching slant-range (SR) SAR images acquired from different orbit and side-viewing directions. However, currently existing algorithms for the SR SAR images matching are unsatisfactory and the correct homologous points cannot be matched. In this work, a robust and adaptable high-precision method for matching flipped SAR images based on oriented descriptors is proposed. Firstly, the ratio gradient calculation based on Gabor odd filter is introduced. Secondly, the oriented descriptors are built to eliminate the diversity of feature-based descriptors of homologous points of flipped SR SAR images, enabling directly match the flipped SAR images. Finally, the feasibility and reliability of the proposed method are verified by the experimental results of simulated flipped SAR images and actual flipped SAR images, and two directional descriptors of the slave image are constructed to match non-flipped SAR images. The results show that the proposed method performs better in precision with the root mean square error (RMSE) less than 1 pixel, and the proposed oriented descriptors can match SR SAR images.

Spatial Pyramid Attention Enhanced Visual Descriptors for Landmark Retrieval

Landmark retrieval, which aims to search for landmark images similar to a query photo within a massive image database, has received considerable attention for many years. Despite this, finding landmarks quickly and accurately still presents some unique challenges. To tackle these challenges, we present a deep learning model, called the Spatial-Pyramid Attention network (SPA). This network is an end-to-end convolutional network, incorporating a spatial-pyramid attention layer that encodes the input image, leveraging the spatial pyramid structure to highlight regional features based on their relative spatial distinctiveness. An image descriptor is then generated by aggregating these regional features. According to our experiments on benchmark datasets including Oxford5k, Paris6k, and Landmark-100, our proposed model, SPA, achieves mean Average Precision (mAP) accuracy of 85.3% with the Oxford dataset, 89.6% with the Paris dataset, and 80.4% in the Landmark-100 dataset, outperforming existing state-of-theart deep image retrieval models.

An emotion recognition system: bridging the gap between humans-machines interaction

Human emotion recognition has emerged as a vital research area in recent years due to its widespread applications in psychology, healthcare, education, entertainment, and human-robot interaction. This research article comprehensively analyzes a machine learning-based six-emotion classification algorithm, focusing on its development, evaluation, and potential applications. The study aims to assess the algorithm's performance, identify its limitations, and discuss the importance of selecting appropriate image descriptors for accurate emotion classification. The algorithm achieved an overall accuracy of 92.23\%, showcasing its potential in various fields. However, the classification of specific emotions, particularly ``excited'' and ``afraid'', demonstrated some limitations, suggesting further refinement. The study also highlights the significance of choosing suitable image descriptors, with the manual distance calculation used in the framework proving effective. This article offers insights into developing and evaluating a six-emotion classification algorithm using a machine learning framework, emphasizing its strengths, limitations, and possible applications in multiple domains. The findings contribute to ongoing efforts in creating robust, accurate, and versatile emotion recognition systems that cater to the diverse needs of various applications across psychology, healthcare, robotics, education, and entertainment.

LMFD: lightweight multi-feature descriptors for image stitching

Image stitching is a fundamental pillar of computer vision, and its effectiveness hinges significantly on the quality of the feature descriptors. However, the existing feature descriptors face several challenges, including inadequate robustness to noise or rotational transformations and limited adaptability during hardware deployment. To address these limitations, this paper proposes a set of feature descriptors for image stitching named Lightweight Multi-Feature Descriptors (LMFD). Based on the extensive extraction of gradients, means, and global information surrounding the feature points, feature descriptors are generated through various combinations to enhance the image stitching process. This endows the algorithm with formidable rotational invariance and noise resistance, thereby improving its accuracy and reliability. Furthermore, the feature descriptors take the form of binary matrices consisting of 0s and 1s, not only facilitating more efficient hardware deployment but also enhancing computational efficiency. The utilization of binary matrices significantly reduces the computational complexity of the algorithm while preserving its efficacy. To validate the effectiveness of LMFD, rigorous experimentation was conducted on the Hpatches and 2D-HeLa datasets. The results demonstrate that LMFD outperforms state-of-the-art image matching algorithms in terms of accuracy. This empirical evidence solidifies the superiority of LMFD and substantiates its potential for practical applications in various domains.

Efficient Analysis of Large-Size Bio-Signals Based on Orthogonal Generalized Laguerre Moments of Fractional Orders and Schwarz–Rutishauser Algorithm

Orthogonal generalized Laguerre moments of fractional orders (FrGLMs) are signal and image descriptors. The utilization of the FrGLMs in the analysis of big-size signals encounters three challenges. First, calculating the high-order moments is a time-consuming process. Second, accumulating numerical errors leads to numerical instability and degrades the reconstructed signals’ quality. Third, the QR decomposition technique is needed to preserve the orthogonality of the higher-order moments. In this paper, the authors derived a new recurrence formula for calculating the FrGLMs, significantly reducing the computational CPU times. We used the Schwarz–Rutishauser algorithm as an alternative to the QR decomposition technique. The proposed method for computing FrGLMs for big-size signals is accurate, simple, and fast. The proposed algorithm has been tested using the MIT-BIH arrhythmia benchmark dataset. The results show the proposed method’s superiority over existing methods in terms of processing time and reconstruction capability. Concerning the reconstructed capability, it has achieved superiority with average values of 25.3233 and 15.6507 with the two metrics PSNR and MSE, respectively. Concerning the elapsed reconstruction time, it also achieved high superiority with an efficiency gain of 0.8. The proposed method is suitable for utilization in the Internet of Healthcare Things.