Image Feature Similarity Research Articles

A Multi-Modal Retrieval Model for Mathematical Expressions Based on ConvNeXt and Hesitant Fuzzy Set

Mathematical expression retrieval is an essential component of mathematical information retrieval. Current mathematical expression retrieval research primarily targets single modalities, particularly text, which can lead to the loss of structural information. On the other hand, multimodal research has demonstrated promising outcomes across different domains, and mathematical expressions in image format are adept at preserving their structural characteristics. So we propose a multi-modal retrieval model for mathematical expressions based on ConvNeXt and HFS to address the limitations of single-modal retrieval. For the image modal, mathematical expression retrieval is based on the similarity of image features and symbol-level features of the expression, where image features of the expression image are extracted by ConvNeXt, while symbol-level features are obtained by the Symbol Level Features Extraction (SLFE) module. For the text modal, the Formula Description Structure (FDS) is employed to analyze expressions and extract their attributes. Additionally, the application of the Hesitant Fuzzy Set (HFS) theory facilitates the computation of hesitant fuzzy similarity between mathematical queries and candidate expressions. Finally, Reciprocal Rank Fusion (RRF) is employed to integrate rankings from image modal and text modal retrieval, yielding the ultimate retrieval list. The experiment was conducted on the publicly accessible ArXiv dataset (containing 592,345 mathematical expressions) and the NTCIR-mair-wikipedia-corpus (NTCIR) dataset.The MAP@10 values for the multimodal RRF fusion approach are recorded as 0.774. These substantiate the efficacy of the multi-modal mathematical expression retrieval approach based on ConvNeXt and HFS.

Graph based encrypted malicious traffic detection with hybrid analysis of multi-view features

At present, the TLS cryptographic protocol is widely deployed. While protecting the security and integrity of transmitted information, it also makes the detection of malicious behavior more difficult. In recent years, researchers have proposed many encrypted malicious traffic detection methods. However, the existing approaches have some shortcomings. Firstly, although researchers have extracted multi-view features from different aspects, which can be divided into vectorized features based on feature engineering and image features based on original data, existing methods cannot fully integrate the features of different forms of expression. Secondly, most of the existing methods do not fully analyze the correlation between different encrypted traffic. Thirdly, the existing methods based on correlation analysis have low processing efficiency and cannot be applied to real networks. In the paper, we present MalDiscovery, a novel technique to discover encrypted malicious traffic to address all the above issues. For encrypted malicious traffic, MalDiscovery constructs an attribute KNN graph, in which encrypted sessions are used as nodes to construct a KNN graph according to the similarity of image features, and vectorized features are used as attributes of corresponding nodes. After that, the GraphSAGE model is used to collect relevant node information through correlation analysis to enrich the embeddings of each node. Finally, we achieve the accurate binary classification of nodes in the graph based on richer embeddings. We use extensive experiments to evaluate the proposed method, and the experiment results show that MalDiscovery can achieve an accuracy of about 99.9%, significantly outperforming all compared methods.

Loop Closure Detection Based on Residual Network and Capsule Network for Mobile Robot.

Loop closure detection based on a residual network (ResNet) and a capsule network (CapsNet) is proposed to address the problems of low accuracy and poor robustness for mobile robot simultaneous localization and mapping (SLAM) in complex scenes. First, the residual network of a feature coding strategy is introduced to extract the shallow geometric features and deep semantic features of images, reduce the amount of image noise information, accelerate the convergence speed of the model, and solve the problems of gradient disappearance and network degradation of deep neural networks. Then, the dynamic routing mechanism of the capsule network is optimized through the entropy peak density, and a vector is used to represent the spatial position relationship between features, which can improve the ability of image feature extraction and expression to optimize the overall performance of networks. Finally, the optimized residual network and capsule network are fused to retain the differences and correlations between features, and the global feature descriptors and feature vectors are combined to calculate the similarity of image features for loop closure detection. The experimental results show that the proposed method can achieve loop closure detection for mobile robots in complex scenes, such as view changes, illumination changes, and dynamic objects, and improve the accuracy and robustness of mobile robot SLAM.

Generation and Evaluation of Synthetic Computed Tomography (CT) from Cone-Beam CT (CBCT) by Incorporating Feature-Driven Loss into Intensity-Based Loss Functions in Deep Convolutional Neural Network.

Simple SummaryDespite numerous benefits of cone-beam computed tomography (CBCT), its applications to radiotherapy were limited mainly due to degraded image quality. Recently, enhancing the CBCT image quality by generating synthetic CT image by deep convolutional neural network (CNN) has become frequent. Most of the previous works, however, generated synthetic CT with simple, classical intensity-driven loss in network training, while not specifying a full-package of verifications. This work trained the network by combining feature- and intensity-driven losses and attempted to demonstrate clinical relevance of the synthetic CT images by assessing both image similarity and dose calculating accuracy throughout a commercial Monte-Carlo.Deep convolutional neural network (CNN) helped enhance image quality of cone-beam computed tomography (CBCT) by generating synthetic CT. Most of the previous works, however, trained network by intensity-based loss functions, possibly undermining to promote image feature similarity. The verifications were not sufficient to demonstrate clinical applicability, either. This work investigated the effect of variable loss functions combining feature- and intensity-driven losses in synthetic CT generation, followed by strengthening the verification of generated images in both image similarity and dosimetry accuracy. The proposed strategy highlighted the feature-driven quantification in (1) training the network by perceptual loss, besides L1 and structural similarity (SSIM) losses regarding anatomical similarity, and (2) evaluating image similarity by feature mapping ratio (FMR), besides conventional metrics. In addition, the synthetic CT images were assessed in terms of dose calculating accuracy by a commercial Monte-Carlo algorithm. The network was trained with 50 paired CBCT-CT scans acquired at the same CT simulator and treatment unit to constrain environmental factors any other than loss functions. For 10 independent cases, incorporating perceptual loss into L1 and SSIM losses outperformed the other combinations, which enhanced FMR of image similarity by 10%, and the dose calculating accuracy by 1–2% of gamma passing rate in 1%/1mm criterion.

BPI-MVQA: a bi-branch model for medical visual question answering

BackgroundVisual question answering in medical domain (VQA-Med) exhibits great potential for enhancing confidence in diagnosing diseases and helping patients better understand their medical conditions. One of the challenges in VQA-Med is how to better understand and combine the semantic features of medical images (e.g., X-rays, Magnetic Resonance Imaging(MRI)) and answer the corresponding questions accurately in unlabeled medical datasets.MethodWe propose a novel Bi-branched model based on Parallel networks and Image retrieval for Medical Visual Question Answering (BPI-MVQA). The first branch of BPI-MVQA is a transformer structure based on a parallel network to achieve complementary advantages in image sequence feature and spatial feature extraction, and multi-modal features are implicitly fused by using the multi-head self-attention mechanism. The second branch is retrieving the similarity of image features generated by the VGG16 network to obtain similar text descriptions as labels.ResultThe BPI-MVQA model achieves state-of-the-art results on three VQA-Med datasets, and the main metric scores exceed the best results so far by 0.2%, 1.4%, and 1.1%.ConclusionThe evaluation results support the effectiveness of the BPI-MVQA model in VQA-Med. The design of the bi-branch structure helps the model answer different types of visual questions. The parallel network allows for multi-angle image feature extraction, a unique feature extraction method that helps the model better understand the semantic information of the image and achieve greater accuracy in the multi-classification of VQA-Med. In addition, image retrieval helps the model answer irregular, open-ended type questions from the perspective of understanding the information provided by images. The comparison of our method with state-of-the-art methods on three datasets also shows that our method can bring substantial improvement to the VQA-Med system.

Image Retrieval Technology of Smart Archives from the Perspective of National Reading

In order to improve the image retrieval effect of smart archives, reduce retrieval time, and enhance retrieval performance, this paper proposes an image retrieval technology for smart archives from the perspective of national reading. According to the concept of nationwide reading and considering the internal and external influencing factors, mobile visual retrieval service for users of smart archives is provided, and the mobile visual retrieval model for users of smart archives is constructed by using a standardized structure and mixed structure. During the operation of the model, the image feature similarity of the wisdom archive is calculated, and the image is processed by the corresponding module, so as to realize the image retrieval technology research of the wisdom archive from the national reading perspective. The experimental results show that the designed technology has better retrieval effect, shorter time, and better performance.

Feature similarity measurement of cross-age face images based on a deep learning algorithm

In order to overcome the problem of low accuracy of traditional facial image feature similarity measurement methods, the paper proposes a new cross-age facial image feature similarity measurement method based on a deep learning algorithm. Face segmentation is performed based on eye coordinates to determine the effective area for face detection. This study introduces deep learning algorithms to build the basic architecture of deep learning networks, define network input as face image data containing age and identity tags in a cross-age database, and define output as face image features under constant age, and pass the foundation separately training and cross-age training complete the network training, and calculate the facial features under the same age. The cosine distance method is used to measure the similarity of face image features across ages. The experimental results show that the method has obtained more accurate measurement results in different age data sets, and the highest measurement accuracy is 97.9%.

Industrial visual perception technology in Smart City

In order to study the application effect and function of industrial visual perception technology in smart city, the image processing and quality evaluation system was constructed by using convolutional neural network (CNN) and Internet of things (IoT) technology. The system was simulated, and then the quality performance of image and video obtained by using industrial visual perception technology was processed and analyzed. The results show that in the analysis of image local optimization effect, it is found that the classification performance of all algorithms decreases with the increase of noise, and the performance of local anisotropic mode (LAP) is superior, which has strong robustness to rotation, illumination, and noise. In the analysis of image feature similarity effect, it is found that the chi square distance between Log Gabor features is positively correlated with the degree of image distortion, and the validity of the measurement method is verified. Further analysis of the video processing effect of industrial visual perception technology shows that the video processing effect of test algorithm is significantly better than that of HM16.8 by comparing the distortion performance of the two algorithms with different sequences, with low distortion and significantly improved performance. Therefore, through the research, it is found that the improved CNN algorithm is superior to other algorithms in image and video processing.

A stereoscopic image quality assessment model based on independent component analysis and binocular fusion property

Quality assessment for stereoscopic image is an important research issue in three-dimensional researches. In this paper, an independent component analysis(ICA) and binocular combination-based full reference image quality assessment (FR-IQA) method is proposed for color stereoscopic image. Specifically, image features that reflect the responds of simple cells in the cortex are first extracted by an ICA-based feature detector, which models the functions of the receptive fields of simple cells in the primary visual cortex. Both image feature similarity (IFS) and local luminance consistency (LLC) of the right and left images are calculated to measure the structure and brightness distortions respectively. Image patches that the mean pixel values have been removed are used for IFS calculation, while the computation of LLC is based on the removed mean pixel value. To simulate the binocular fusion properties of the complex cells in the cortex, feature energy of the extracted image features is utilized to calculate the weighting factors of binocular combination, following which a single feature similarity index is obtained by fusing the right and left images IFS. Furthermore, global relative luminance information of the selected image patches is used to integrate the right and left images LLC into a single luminance consistency index. Finally, image quality is obtained by combining above two indexes. Experimental results demonstrate that the proposed algorithm achieves high consistency with subjective assessment on 3D image quality assessment databases.

DeepProp: Extracting Deep Features from a Single Image for Edit Propagation

AbstractEdit propagation is a technique that can propagate various image edits (e.g., colorization and recoloring) performed via user strokes to the entire image based on similarity of image features. In most previous work, users must manually determine the importance of each image feature (e.g., color, coordinates, and textures) in accordance with their needs and target images. We focus on representation learning that automatically learns feature representations only from user strokes in a single image instead of tuning existing features manually. To this end, this paper proposes an edit propagation method using a deep neural network (DNN). Our DNN, which consists of several layers such as convolutional layers and a feature combiner, extracts stroke‐adapted visual features and spatial features, and then adjusts the importance of them. We also develop a learning algorithm for our DNN that does not suffer from the vanishing gradient problem, and hence avoids falling into undesirable locally optimal solutions. We demonstrate that edit propagation with deep features, without manual feature tuning, can achieve better results than previous work.

Mathematical problems of joint reconstruction for biomedical multi-modality imaging techniques

Biomedical imaging aims at visualizing structural or functional information necessary for biological research or clinical diagnosis. Mathematics has played a significant role in biomedical imaging technology. A class of mathematical problems in biomedical imaging techniques is to reconstruct the information of interest from measured data, which typically poses an ill-posed inverse problem. One recent development is the multi-modality imaging technique for enriching image information by fusing multiple imaging modalities. In contrast to the existing single-mode imaging technology, there is a variety of strategies for image reconstruction of multi-modality imaging techniques. A new direction is the use of the similarity of multi-modality images, or more exactly, the use of the partial similarity of image features from different modality images for achieving the joint reconstruction of multi-modality images. The advantages of joint reconstruction are to utilize the data from different modalities to steer the reconstruction of one modality through the complementary feature information from another modality and to get better quality images. This note is a review of mathematical problems of the joint reconstruction for multi-modality imaging techniques. We introduce the key mathematical problems and report relevant progresses in this field. The fundamental Mumford-Shah functional in this approach has been discoursed in a book by Zhang Gongqing.

Dissociating the effects of angular disparity and image similarity in mental rotation and object recognition

Performance is often impaired linearly with increasing angular disparity between two objects in tasks that measure mental rotation or object recognition. But increased angular disparity is often accompanied by changes in the similarity between views of an object, confounding the impact of the two factors in these tasks. We examined separately the effects of angular disparity and image similarity on handedness (to test mental rotation) and identity (to test object recognition) judgments with 3-D novel objects. When similarity was approximately equated, an effect of angular disparity was only found for handedness but not identity judgments. With a fixed angular disparity, performance was better for similar than dissimilar image pairs in both tasks, with a larger effect for identity than handedness judgments. Our results suggest that mental rotation involves mental transformation procedures that depend on angular disparity, but that object recognition is predominately dependent on the similarity of image features.

A Robust and Fast Non-Local Means Algorithm for Image Denoising

In the paper, we propose a robust and fast image denoising method. The approach integrates both 'Non-Local means algorithm and Laplacian Pyramid. Given an image to be denoised, we first decompose it into Laplacian pyramid. Exploiting the redundancy property of Laplacian pyramid, we then perform non-local means on every level image of Laplacian pyramid. Essentially, we use the similarity of image features in Laplacian pyramid to act as weight to denoise image. Since the features extracted in Laplacian pyramid are localized in spatial position and scale, they are much more able to describe image, and computing the similarity between them is more reasonable and more robust. Also, based on the efficient Summed Square Image (SSI) scheme and Fast Fourier Transform (FFT), we present an accelerating algorithm to break the bottleneck of non-local means algorithm -similarity computation of compare windows. After speedup, our algorithm is fifty times faster than original non-local means algorithm. Experiments demonstrated the effectiveness of our algorithm.