Multiple Semantic Features Research Articles

Hierarchical visual-semantic interaction for scene text recognition

Proper interaction between visual and semantic features is crucial to obtain a powerful feature representation for scene text recognition (STR). The existing interaction methods usually treat visual and semantic features as distinct tokens and use transformers to learn contextual information and prior language knowledge, and they achieve promising performance for STR task. However, there still remain several issues needed to be further addressed such as the imbalance number and mis-alignment between visual and semantic features, and the necessarily of stacking transformers to a progressive improvement in accuracy. To this aim, this paper proposes a novel interaction manner namely hierarchical visual-semantic interaction (HVSI) which contains three novel modules including a hierarchical visual-semantic interaction module, fusion module, and visual-semantic alignment module. The hierarchical visual-semantic interaction module employs multiple visual-semantic interaction blocks in various scales to enhance the representation power of visual features and semantic features. To better exploit multi-scale visual and semantic features, the fusion module is introduced to fuse multiple semantic features based on attention mechanisms. Furthermore, our HVSI presents a simple plug-in block named visual-semantic alignment module to alleviate mis-alignment of semantic features by mapping them into a unified semantic space, which helps improve the performance of HVSI. Extensive experiments on multiple benchmarks including English and Chinese text recognition datasets show that our method obtains state-of-the-art or competitive performances.

Multi-semantic feature fusion attention network for binary code similarity detection

Binary code similarity detection (BCSD) plays a big role in the process of binary application security test. It can be applied in several fields, such as software plagiarism detection, malware analysis, vulnerability detection. Most research is based on recurrent neural networks, which is difficult to get the overall or long-distance semantic information of functions. Besides, exiting works simply extract high-level semantic features, lacking in-depth investigations on the potential mechanisms for fusing low-level and high-level semantic features. In this paper we propose a multi-semantic feature fusion attention network (MFFA-Net) for BCSD. MFFA-Net contains two critical modules: semantic feature fusion (SFF) and attention feature fusion (AFF). The SFF module concatenates multiple semantic features to represent the semantics of the function, which helps to obtain the overall semantic information of the function. The AFF module is designed to find useful information from various features, which assigns an attention matrix to research the relationship between features. In order to evaluate the proposed method, we made extensive experiments on two datasets. MFFA-Net can achieve a high degree of AUC at 99.6% and 98.3% respectively on the two datasets. The experimental results show that MFFA-Net has better performance for BCSD.

Agricultural Text Classification Method Based on Dynamic Fusion of Multiple Features

The traditional text classification method, which treats the values in agricultural text as characters, will lose the original semantic expression of numerical features. In order to fully mine the deep latent semantic features in agricultural text, a novel text classification method based on multivariate feature dynamic fusion is proposed. The Bi-directional Long Short Term Memory network (Bi-LSTM) model with attention mechanism was used to extract the global key semantic features of the text; the multiple windows Convolution Neural Network were used to extract the local semantic information of the text at different levels; Numerical value features containing essential semantic expression were extracted by artificial method to construct the numerical value feature vector. By introducing the attention mechanism to dynamically fuse the extracted multiple semantic features, which can further enrich the deep semantic expression of agricultural text and effectively improve the effect of agricultural text classification with phenotypic numerical type.

A Deep Learning Method for Android Application Classification Using Semantic Features

Android has become the most popular mobile intelligent operating system with its open platform, diverse applications, and excellent user experience. However, at the same time, more and more attackers take Android as the primary target. The application store, which is the main download source for users, still does not have a complete security authentication mechanism. Given the above problems, we designed an Android application classification model based on multiple semantic features. Firstly, we use analysis tools to automatically extract the application’s dynamic and static features into the text document and use variance and chi-square tests to optimize the features. Combined with natural language processing (NLP), we transform the feature file into a two-dimensional matrix and use the convolution neural network (CNN) to learn features efficiently. Also, to make the model satisfy more application scenarios, we design a dynamic adjustment method according to user requirements, the number of features, and other indicators. The experimental results demonstrate that the detection accuracy of malware is 99.3921%. We also measure this model’s performance in detecting a malware family and benign application, with the classification accuracy of 99.5614% and 99.9046%, respectively.

Learning Motivations and Learning Behaviors of Sports Majors Based on Big Data

The subjective factors of sports majors play a critical role in the improvement of their cultural quality. Based on data mining, the valuable information about learning motivation and learning behavior can be obtained from the massive data. Therefore, this paper explores the learning motivations and learning behaviors of sports majors based on big data. Firstly, this paper analyzed the features of the learning behaviors of sports majors, and measured the complexity of their learning behaviors with information entropy, approximate entropy, and change-complexity function. Next, a dataset was established based on the students’ use of campus access network and online learning platforms. After that, a time domain convolutional capsule network model of multiple semantic features was established to recognize and classify the learning motivations of sports majors. The proposed model was proved effective through experiments.

Feature pyramid random fusion network for visible-infrared modality person re-identification

Existing works in person re-identification only considers extracting invariant feature representations from cross-view visible cameras, which ignores the imaging feature in infrared domain, such that there are few studies on visible-infrared relevant modality. Besides, most works distinguish two-views by often computing the similarity in feature maps from one single convolutional layer, which causes a weak performance of learning features. To handle the above problems, we design a feature pyramid random fusion network (FPRnet) that learns discriminative multiple semantic features by computing the similarities between multi-level convolutions when matching the person. FPRnet not only reduces the negative effect of bias in intra-modality, but also balances the heterogeneity gap between inter-modality, which focuses on an infrared image with very different visual properties. Meanwhile, our work integrates the advantages of learning local and global feature, which effectively solves the problems of visible-infrared person re-identification. Extensive experiments on the public SYSU-MM01 dataset from aspects of mAP and convergence speed, demonstrate the superiorities in our approach to the state-of-the-art methods. Furthermore, FPRnet also achieves competitive results with 32.12% mAP recognition rate and much faster convergence.

MVANet: Multi-Task Guided Multi-View Attention Network for Chinese Food Recognition

Food recognition plays a much critical role in various health-care applications. However, it poses many challenges to current approaches due to the diverse appearances of food dishes and the non-uniform composition of ingredients for the foods in the same category. Current methods primarily focus on the appearance of foods without considering their semantic information, easily finding the wrong attention areas of food images. Second, these methods lack the dynamic weighting of multiple semantic features in the modeling process. Thus this paper proposes a novel Multi-View Attention Network within the multi-task learning framework that incorporates multiple semantic features into the food recognition task from both ingredient recognition and recipe modeling. It also utilizes the multi-view attention mechanism to automatically adjust the weights of different semantic features and enables different tasks to interact with each other so as to obtain a more comprehensive feature representation. The experiments conducted on both ChineseFoodNet and VIREO Food-172 benchmark databases validate the proposed method with the obvious improvement of the performance and the lower parameter size.

Learning in complex, multi-component cognitive systems: Different learning challenges within the same system.

Using word learning as an example of a complex system, we investigated how differences in the structure of the subcomponents in which learning occurs can have significant consequences for the challenge of integrating new information within such systems. Learning a new word involves integrating information into the two key stages/subcomponents of processing within the word production system. In the first stage, multiple semantic features are mapped onto a single word. Conversely, in the second stage, a single word is mapped onto multiple segmental features. We tested whether the unitary goal of word learning leads to different local outcomes in these two stages because of their reversed mapping patterns. Neurotypical individuals (N = 17) learned names and semantic features for pictures of unfamiliar objects presented in semantically related, segmentally related and unrelated blocks. Both similarity types interfered with word learning. However, feature learning was differentially affected within the two subcomponents of word production. Semantic similarity facilitated learning distinctive semantic features (i.e., features unique to each item), whereas segmental similarity facilitated learning shared segmental features (i.e., features common to several items in a block). These results are compatible with an incremental learning model in which learning not only strengthens certain associations but also weakens others according to the local goals of each subcomponent. More generally, they demonstrate that the same overall learning goal can lead to opposite learning outcomes in the subcomponents of a complex system. The general principles uncovered may extend beyond word learning to other complex systems with multiple subcomponents. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

The Effect of Training-Induced Visual Imageability on Electrophysiological Correlates of Novel Word Processing.

The concreteness effect (CE) describes a processing advantage for concrete over abstract words. Electrophysiologically, the CE manifests in higher N400 and N700 amplitudes for concrete words. The contribution of the stimulus-inherent imageability to the electrophysiological correlates of the CE is not yet fully unraveled. This EEG study focused on the role of imageability irrespective of concreteness by examining the effects of training-induced visual imageability on the processing of novel words. In two training sessions, 21 healthy participants learned to associate novel words with pictures of novel objects as well as electron-microscopical structures and were additionally familiarized with novel words without any picture association. During a post-training EEG session, participants categorized trained novel words with or without picture association, together with real concrete and abstract words. Novel words associated with novel object pictures during the training elicited a higher N700 than familiarized novel words without picture-association. Crucially, this training-induced N700 effect resembled the CE found for real words. However, a CE on the N400 was found for real words, but no effect of imageability in novel words. The results suggest that the N400 CE for real words depends on the integration of multiple semantic features, while mere visual imageability might contribute to the CE in the N700 time window.

Selective Audiovisual Semantic Integration Enabled by Feature-Selective Attention.

An audiovisual object may contain multiple semantic features, such as the gender and emotional features of the speaker. Feature-selective attention and audiovisual semantic integration are two brain functions involved in the recognition of audiovisual objects. Humans often selectively attend to one or several features while ignoring the other features of an audiovisual object. Meanwhile, the human brain integrates semantic information from the visual and auditory modalities. However, how these two brain functions correlate with each other remains to be elucidated. In this functional magnetic resonance imaging (fMRI) study, we explored the neural mechanism by which feature-selective attention modulates audiovisual semantic integration. During the fMRI experiment, the subjects were presented with visual-only, auditory-only, or audiovisual dynamical facial stimuli and performed several feature-selective attention tasks. Our results revealed that a distribution of areas, including heteromodal areas and brain areas encoding attended features, may be involved in audiovisual semantic integration. Through feature-selective attention, the human brain may selectively integrate audiovisual semantic information from attended features by enhancing functional connectivity and thus regulating information flows from heteromodal areas to brain areas encoding the attended features.

基于视频语料库的多义词「切る」的认知语义研究

有关「切る」的先行研究很多，但基于视频语料库以认知语义学的原理研究多义词「切る」的尚未发现。本文基于《JV Finder》视频语料库，通过隐喻和转喻两个观点入手，探讨多义词「切る」的语义特征，并对义项之间的关联性进行阐释。除了对本动词「切る」进行分析之外，也列举相关的复合动词，对其扩展义进行周详解释。比较抽象的惯用句和“不可分解型”的复合动词(語彙的複合動詞)的学习，建议结合视频语料库的多媒体立体语境提供的信息，加深多义词「切る」的语义扩张的理解。考察结果发现「切る」的扩展义在日剧口语对白里大部分是隐喻用法。 While many researchers have focused their studies on the Japanese word “Kiru”, audio-visual corpus-based cognitive semantics studies on its extended meanings are rare. Based on “JV Finder”, an audio-visual corpus, the current article addresses how multiple semantic features of the word “Kiru” are formed and related to one another from both the metaphor and metonymy aspects. The article also addresses in details the extended meanings of this word by presenting rich analyses of it and examples of its relevant compound verbs. Findings suggest that using context-based three dimensional information provided by the audio-visual corpus contributes very positively to understanding the extended meanings of abstract idiomatic expressions and/or fixed compound verbs related to this word. The article concludes that the extended meanings of the word “Kiru” used in conversations are, mostly, metaphors.

Autonomous Framework for Sensor Network Quality Annotation: Maximum Probability Clustering Approach

In this paper an autonomous feature clustering framework has been proposed for performance and reliability evaluation of an environmental sensor network. Environmental time series were statistically preprocessed to extract multiple semantic features. A novel hybrid clustering framework was designed based on Principal Component Analysis (PCA), Guided Self-Organizing Map (G-SOM), and Fuzzy-C-Means (FCM) to cluster the historical multi-feature space into probabilistic state classes. Finally a dynamic performance annotation mechanism was developed based on Maximum (Bayesian) Probability Rule (MPR) to quantify the performance of an individual sensor node and network. Based on the results from this framework, a “data quality knowledge map” was visualized to demonstrate the effectiveness of this framework.

Can segregation within the semantic system account for category-specific deficits?

Functional neuroimaging was used to investigate the extent to which category-specific semantic deficits in patients can be accounted for in terms of the demands placed on neural systems underlying different types of semantic knowledge. Unlike previous functional imaging studies of category specificity, we used a factorial design that crossed category (tools and fruits) with tasks requiring retrieval of either action or perceptual (real life size) knowledge. The presentation of tools relative to fruit increased activation in the same left posterior middle temporal area that was linked to the retrieval of action knowledge in general (for fruit as well as tools). However, we found no correlation between activation evoked by fruit and the size retrieval task. The left medial anterior temporal cortex was the only region to be activated for fruit relative to tools. We argue that the sensory-functional theory of category-specific effects is insufficient to account for the current neuroimaging literature. However, the data do support a more refined version of the theory: tools, relative to fruit, are more strongly linked to manipulative/motor knowledge and, for some tasks, fruit may be more reliant on integrating multiple semantic features.