Local Semantic Features Research Articles

LUN-BiSeNetV2: A lightweight unstructured network based on BiSeNetV2 for road scene segmentation

With the continuous introduction of automatic driving technology, the research of road scene segmentation algorithm in machine vision has become very important. In traditional methods, most researchers use machine learning methods to segment thresholds. However, the introduction of deep learning in recent years makes convolutional neural networks widely used in this field. Aiming at the problem that the traditional threshold segmentation method is difficult to effectively extract the threshold value of road image in multiple scenes and the serious problem of over-segmentation caused by deep neural network training data directly, this paper proposes a road scene segmentation method based on a lightweight unstructured network based on BiSeNetV2. The network contains backbone segmentation network and BiSeNetV2 network. The Mobilenetv2 network is used in the backbone network to replace the Xception feature extraction network in the decoder. In addition, grouping convolution is used to replace common convolution in Mobilenetv2 network. And it selects the batch specification layer to reduce the number of parameters, without affecting the accuracy and improving the efficiency of segmentation. At the same time, due to the relatively fixed distribution position of unstructured roads in the image, attention mechanism is introduced to process advanced semantic features, so as to improve the sensitivity and accuracy of the network. The BiSeNetV2 network enhances the dominant relationship between channel features by adding a compression excitation module based on channel attention mechanism after the detail branch, so as to perceive key areas and highlight local features. The lightweight feature pyramid attention mechanism is used to optimize semantic branches, improve the feature integration between contexts, extract high-level road semantic information more efficiently and retain spatial location information to the maximum extent. Finally, local semantic features and high-level semantic features are fused to improve the effect of unstructured road detection. The experiment is trained on the open data set. The results show that compared with other state-of-theart networks, the accuracy and real-time performance of proposed LUN-BiSeNetV2 in this paper are good, and the false segmentation and edge clarity are better. Compared with the classical algorithm, the average intersection is improved by 2.2% compared with mIoU, the average pixel accuracy is improved by 7.6%, and the frame rate is improved by 24.5%.

CCFNet: Cross-Complementary fusion network for RGB-D scene parsing of clothing images

Schemes to complement context relationships by cross-scale feature fusion have appeared in many RGB-D scene parsing algorithms; however, most of these works conduct multi-scale information interaction after multi-modal feature fusion, which ignores the information loss of the two modes in the original coding. Therefore, a cross-complementary fusion network (CCFNet) is designed in this paper to calibrate the multi-modal information before feature fusion, so as to improve the feature quality of each mode and the information complementarity ability of RGB and the depth map. First, we divided the features into low, middle, and high levels, among which the low-level features contain the global details of the image and the main learning features include texture, edge, and other features. The middle layer features contain not only some global detail features but also some local semantic features. Additionally, the high-level features contain rich local semantic features. Then, the feature information lost in the coding process of low and middle level features is supplemented and extracted through the designed cross feature enhancement module, and the high-level features are extracted through the feature enhancement module. In addition, the cross-modal fusion module is designed to integrate multi-modal features of different levels. The experimental results verify that the proposed CCFNet achieves excellent performance on the RGB-D scene parsing dataset containing clothing images, and the generalization ability of the model is verified by the dataset NYU Depth V2.

MAGE: Multi-scale Context-aware Interaction based on Multi-granularity Embedding for Chinese Medical Question Answer Matching

Background and ObjectiveThe Chinese medical question answer matching (cMedQAM) task is the essential branch of the medical question answering system. Its goal is to accurately choose the correct response from a pool of candidate answers. The relatively effective methods are deep neural network-based and attention-based to obtain rich question-and-answer representations. However, those methods overlook the crucial characteristics of Chinese characters: glyphs and pinyin. Furthermore, they lose the local semantic information of the phrase by generating attention information using only relevant medical keywords. To address this challenge, we propose the multi-scale context-aware interaction approach based on multi-granularity embedding (MAGE) in this paper. MethodsWe adapted ChineseBERT, which integrates Chinese characters glyphs and pinyin information into the language model and fine-tunes the medical corpus. It solves the common phenomenon of homonyms in Chinese. Moreover, we proposed a context-aware interactive module to correctly align question and answer sequences and infer semantic relationships. Finally, we utilized the multi-view fusion method to combine local semantic features and attention representation. ResultsWe conducted validation experiments on the three publicly available datasets, namely cMedQA V1.0, cMedQA V2.0, and cEpilepsyQA. The proposed multi-scale context-aware interaction approach based on the multi-granularity embedding method is validated by top-1 accuracy. On cMedQA V1.0, cMedQA V2.0, and cEpilepsyQA, the top-1 accuracy on the test dataset was improved by 74.1%, 82.7%, and 60.9%, respectively. Experimental results on the three datasets demonstrate that our MAGE achieves superior performance over state-of-the-art methods for the Chinese medical question answer matching tasks. ConclusionsThe experiment results indicate that the proposed model can improve the accuracy of the Chinese medical question answer matching task. Therefore, it may be considered a potential intelligent assistant tool for the future Chinese medical answer question system.

A Text Classification Model via Multi-Level Semantic Features

Text classification is a major task of NLP (Natural Language Processing) and has been the focus of attention for years. News classification as a branch of text classification is characterized by complex structure, large amounts of information and long text length, which in turn leads to a decrease in the accuracy of classification. To improve the classification accuracy of Chinese news texts, we present a text classification model based on multi-level semantic features. First, we add the category correlation coefficient to TF-IDF (Term Frequency-Inverse Document Frequency) and the frequency concentration coefficient to CHI (Chi-Square), and extract the keyword semantic features with the improved algorithm. Then, we extract local semantic features with TextCNN with symmetric-channel and global semantic information from a BiLSTM with attention. Finally, we fuse the three semantic features for the prediction of text categories. The results of experiments on THUCNews, LTNews and MCNews show that our presented method is highly accurate, with 98.01%, 90.95% and 94.24% accuracy, respectively. With model parameters two magnitudes smaller than Bert, the improvements relative to the baseline Bert+FC are 1.27%, 1.2%, and 2.81%, respectively.

Joint Event Relation Identification Based on Multiscale Convolutional Neural Network and Sharing Strategy

At present, most of the event relation identification work mainly focuses on the sequential temporal and explicit causal relation between events. These methods usually ignore the role of synchronous temporal and implicit causal relation in sentences, which makes the semantic understanding of the model deviate from the text. In this paper, we propose a joint event relation identification model. The model uses bidirectional GRU and multiscale convolutional neural network to obtain the context semantic features and multiscale local semantic features of text, respectively. Then, these two kinds of features are fused to fully obtain the semantics of the text itself. In addition, we build encoders and decoders of event temporal and causal relation, respectively, to obtain the event temporal and causal semantic features from text. In this process, considering the correlation between event timing and causality, we use three different parameter sharing strategies to realize the interaction between event temporal and causal semantic features. The experimental results on the legal field dataset we constructed show that our model has made significant improvements compared with the baseline model. Through experimental analysis, our method can effectively improve the identification performance of synchronous temporal and implicit causality relation.

A text classification method based on a convolutional and bidirectional long short-term memory model

Text classification refers to labelling text with specified labels, and it is widely used in public opinion supervision, spam detection, and other fields. However, due to the complex semantics of natural language and the difficulty of extracting semantic features, users of traditional methods encounter difficulties when trying to achieve better classification results. In response to this problem, a text classification method based on the CBM (Convolutional and Bi-LSTM Model) model, which can extract shallow local semantic features and deep global semantic features, is proposed. First, the text is vectorised using the Glove model in the embedding layer. Then, the vector text is sent to the Multiscale Convolutional Neural Network (MCNN) and the Bidirectional Long Short-Term Memory network (Bi-LSTM) respectively. The Bi-LSTM layer is also designed in the present work with use of mixed attention to extract deeper semantic features. Finally, the MCNN features and Bi-LSTM features are fused and sent to the softmax layer for classification. Experimental results show that the model can significantly improve the accuracy of text classification.

Traffic flow prediction based on depthwise separable convolution fusion network

Traffic flow prediction is an important part of an intelligent transportation system to alleviate congestion. In practice, most small and medium-sized activities are not given priority in transport planning, yet these activities often bring about a surge in demand for public transport. It is recognized that such patterns are inevitably more difficult to predict than those associated with day-to-day mobility, and that forecasting models built using traffic data alone are not comprehensive enough. Aiming at this problem, a depthwise separable convolutional fusion forecast network (FFN) was proposed by focusing on the impact of event information on traffic flow demand. FFN fused heterogeneous data to model traffic data, weather information, and event information extracted from the Internet. The depthwise separable one-dimensional convolution was used to encode the textual information describing the event layer by layer, and local one-dimensional sequence segments (ie subsequences) were extracted from the sequence to retain rich local semantic features. In the modeling process, the interaction of heterogeneous data was established, that is, the temporal and other data were used to drive the textual information representation in the encoding process to capture better relevant textual representations. Finally, information from different sources and formats was fused to obtain a joint feature representation tensor that predicts the traffic demand in the next day's event area. The experimental results show that the average absolute error of the fusion prediction network is reduced by 26.5%, the root mean square error is reduced by 11.6%, and the judgment coefficient is increased by 26.4% compared with the prediction network that only considers the traffic data.

Adaptive multi-scale transductive information propagation for few-shot learning

Few-shot learning aims to learn a classifier with more generalization capability from extremely limited labeled samples has drawn an increasing amount of attention in many areas. One typical work in this field is the transductive propagation network (TPN), which propagates labels by capturing the local geometry distribution information between data. However, TPN exploited only the manifold structure of feature extractor networks’ high-layer semantic features (single-scale) distribution, while neglecting their local geometry distribution in the low-layer semantic features. This paper presents the adaptive multi-scale transductive information propagation (AMTIP) model to address this problem. Specifically, we introduce the multi-scale feature extractor networks to simultaneously learn samples’ high-layer global semantic features and low-layer local semantic features. After acquiring the multi-scale features, we propose the adaptive multi-scale fusion networks to generate the adaptive semantic fusion features applying to different few-shot classification tasks. Finally, the adaptive semantic fusion features are applied to the label propagation model for few-shot image classification. Compared with the single-scale semantic features of TPN, AMTIP can better preserve the local geometry between the support set and query set data via the adaptive multi-scale semantic fusion features. Extensive experiments demonstrate the superiority of the proposed AMTIP compared to the state-of-the-art few-shot classification models.

ConvPatchTrans: A script identification network with global and local semantics deeply integrated

Optical Character Recognition (OCR) system serves the need of reading text from images. Script identification that identifies the language of the text in the image is an important part of OCR technology and an indispensable role in the stability and accuracy of the OCR system. The most challenging for script identification is the interference caused by similarities between texts in different languages. In this paper, a two-branch network named ConvPatchTrans is designed to process global and local semantic features separately, focusing on the text and each word in a picture. The ConvPatchTrans extracts feature from different stages of the Visual Geometry Group network (VGGNet) as global and local semantics. For the global branch, the linear classifier is recommended. For the local branch, text image data is converted to image sequence data. Then, multi-layers convolution-enhanced Transformer (MCET) is proposed to bring about the deep fusion of sequence. Finally, the global and local branches are fused by an adaptive weighted fusion method to get the best result. In order to verify the effectiveness of our proposed method, four public script identification datasets are used for comparative experiments. Our method has obtained the highest values among currently published methods on the CVSI2015 and MLE2E datasets, which are 98.90% and 97.50%, respectively. At the same time, satisfactory results are also obtained on the other two datasets.

Lie Group spatial attention mechanism model for remote sensing scene classification

ABSTRACT Utilizing discriminative features to represent data samples is a significant step, and the remote sensing domain is no exception. Most existing convolutional neural network (CNN) models have achieved great results. However, they mainly focus on global high-level features, ignoring local, shallower features and the relationships between features, which are crucial for scene classification. In this study, a novel Lie Group spatial attention mechanism model is introduced. First, it uses Lie Group machine learning and CNN to preserve features at different levels. Then, the Lie Group spatial attention mechanism is used to suppress irrelevant features and enhance local semantic features. Finally, the Lie Fisher classifier is used for prediction. Extensive experiments on two publicly and challenging data sets demonstrate that our model enhances feature characterization capabilities and achieves competitive accuracy with other state-of-the-art methods.

A Combination of Lie Group Machine Learning and Deep Learning for Remote Sensing Scene Classification Using Multi-Layer Heterogeneous Feature Extraction and Fusion

Discriminative feature learning is the key to remote sensing scene classification. Previous research has found that most of the existing convolutional neural networks (CNN) focus on the global semantic features and ignore shallower features (low-level and middle-level features). This study proposes a novel Lie Group deep learning model for remote sensing scene classification to solve the above-mentioned challenges. Firstly, we extract shallower and higher-level features from images based on Lie Group machine learning (LGML) and deep learning to improve the feature representation ability of the model. In addition, a parallel dilated convolution, a kernel decomposition, and a Lie Group kernel function are adopted to reduce the model’s parameters to prevent model degradation and over-fitting caused by the deepening of the model. Then, the spatial attention mechanism can enhance local semantic features and suppress irrelevant feature information. Finally, feature-level fusion is adopted to reduce redundant features and improve computational performance, and cross-entropy loss function based on label smoothing is used to improve the classification accuracy of the model. Comparative experiments on three public and challenging large-scale remote-sensing datasets show that our model improves the discriminative ability of features and achieves competitive accuracy against other state-of-the-art methods.

Aspect-Level Sentiment Analysis with Local Semantic and Global Syntactic Features Integration

Aspect-level sentiment analysis aims to predict the sentiment polarity toward a specific aspect in a sentence. Most current approaches are based on deep learning and the attention mechanism. However, these models cannot simultaneously include the context semantic information carried by local words and the global syntactic information possibly carried by remote words. In this paper, we propose a local semantic and global syntactic integration scheme, which employs a local focus mechanism over local context words and exploits improved graph convolutional networks over dependency tree to encode global syntactic information. Moreover, multi-head attention is used to capture both the semantic information and also the interactive information between semantics and syntactic features. Experimental results on five datasets show the effectiveness of our model over a series of latest models.

LSLPCT: An Enhanced Local Semantic Learning Transformer for 3-D Point Cloud Analysis

The 3D point cloud is a common 3D data representation that has received increasing attention for remote sensing applications. However, processing 3D point cloud semantics, especially local semantic information, has always been a challenge and has attracted much attention. In this paper, we propose a novel enhanced local semantic learning transformer for 3D point cloud analysis, which aims to enhance the transformer awareness of local semantic features to handle complex point cloud tasks. First, we propose a novel transformer framework, the local semantic learning point cloud transformer (LSLPCT), which not only learns 3D point clouds the global information, but also enhances the perception of local semantic information end-to-end. Second, we design an efficient local semantic learning self-attention mechanism, namely LSL-SA, which can parallelize the perception of global contextual information and the capture of finer-grained local semantic features. Third, our proposed LSL-SA is easy to implement and can integrate existing transformers and CNN-based networks for processing various point cloud tasks. Numerous experiments in different types of point cloud tasks have been conducted, and our method performs better or is competitive with other state-of-the-art methods.

Salient-VPR: Salient Weighted Global Descriptor for Visual Place Recognition

Visual place recognition (VPR) is a widely investigated but challenging problem, which must deal with the twin problems of changed appearance caused by varying weather conditions, illumination, and seasons, as well as dynamic objects in complex environments. In this paper, we proposed Salient-VPR by combining image retrieval, semantic information, and saliency cues to achieve accurate estimations. We provides a novel formulation for combining the local semantic features into global descriptors. Unlike traditional global descriptors that typically summarize the whole visual content in the images or image regions, we aggregate dense local semantic descriptors based on pixel-level semantic scores to form global semantic descriptors. Then we calculate predicted saliency descriptors to learn the representation of static objects from images. These salient descriptions are learned from a dataset tailored for VPR missions. Lastly, we introduce a late-fusion module, which increases the stability of the descriptor and avoids performance degradation caused by the limitations of semantic and saliency prediction. Our method outperforms traditional global descriptors in experiments, attaining state-of-the-art visual location recognition scores on a variety of challenging datasets such as Pitts30k, MSLS, and Tokyo24/7.

Микротопонимы д. Янтык Тюменского района Тюменской области

The paper presents the microtoponymy of the village of Yantyk located in the Tyumensky District of Tyumen Oblast. The purpose of the study is to describe the system of microtoponyms of the village of Yantyk located in the Tyumensky District of Tyumen Oblast. Scientific novelty of the study lies in conducting a comprehensive analysis of the microtoponymic system of the settlement. In order to identify the origin of the naming units, the researchers used historical and local lore data, as well as the accounts of the local population clarifying the motivation of the names, which were collected during field research as a part of a comprehensive expedition undertaken by G. Ibragimov Institute of Language, Literature and Art, Tatarstan Academy of Sciences. As a result, it has been proved that the microtoponymy of the village of Yantyk located in the Tyumensky District of Tyumen Oblast reflects both microtoponymic naming units characteristic of the Tatar (Turkic) onomastic system and local semantic, structural and motivational features.

A deceptive reviews detection model: Separated training of multi-feature learning and classification

The increasing online reviews play an essential role in the e-commerce platform, which profoundly affects the purchase decisions of consumers. However, rampant dishonest sellers manipulate other buyers or robots to post deceptive reviews for profit. Recently, the detection of deceptive reviews has attracted general research attention, which mainly comprises two directions, traditional methods based on statistics and intelligent methods based on neural networks. These methods use a single feature or multiple features for classifier design. To make full use of different features for better feature representation of detecting deceptive reviews, this paper proposes a new feature fusion strategy and verifies its performance by comparing it with other feature fusion strategies. First, we utilize three independent models for feature extraction: the TextCNN, the Bidirectional Gated Recurrent Unit (GRU), and the Self-Attention are used to learn local semantic features, temporal semantic features, and weighted semantic features of reviews, respectively. Secondly, after obtaining different feature representations from the fully connected layers of these three models, we concatenate them together to form the final documental representation. Finally, we use a full connection layer and the sigmoid function to further learn and complete deceptive review detection. Experiments on three balanced and unbalanced in-domain small datasets (hotel, restaurant, doctor) and mixed-domain datasets show that our model is superior to baselines. Experiments on large-scale data with various imbalanced proportions verify the effectiveness of our method. We also analyze the results of different datasets from the perspective of part of speech to improve the model's interpretability.

Integrating Normal Vector Features into an Atrous Convolution Residual Network for LiDAR Point Cloud Classification

LiDAR point clouds are rich in spatial information and can effectively express the size, shape, position, and direction of objects; thus, they have the advantage of high spatial utilization. The point cloud focuses on describing the shape of the external surface of the object itself and will not store useless redundant information to describe the occupation. Therefore, point clouds have become the research focus of 3D data models and are widely used in large-scale scene reconstruction, virtual reality, digital elevation model production, and other fields. Since point clouds have various characteristics, such as disorder, density inconsistency, unstructuredness, and incomplete information, point cloud classification is still complex and challenging. To realize the semantic classification of LiDAR point clouds in complex scenarios, this paper proposes the integration of normal vector features into an atrous convolution residual network. Based on the RandLA-Net network structure, the proposed network integrates the atrous convolution into the residual module to extract global and local features of the point clouds. The atrous convolution can learn more valuable point cloud feature information by expanding the receptive field. Then, the point cloud normal vector is embedded in the local feature aggregation module of the RandLA-Net network to extract local semantic aggregation features. The improved local feature aggregation module can merge the deep features of the point cloud and mine the fine-grained information of the point cloud to improve the model’s segmentation ability in complex scenes. Finally, to resolve the imbalance of the distribution of the various categories of point clouds, the original loss function is optimized by adopting a reweighted method to prevent overfitting so that the network can focus on small target categories in the training process to effectively improve the classification performance. Through the experimental analysis of a Vaihingen (Germany) urban 3D semantic dataset from the ISPRS website, it is verified that the proposed algorithm has a strong generalization ability. The overall accuracy (OA) of the proposed algorithm on the Vaihingen urban 3D semantic dataset reached 97.9%, and the average reached 96.1%. Experiments show that the proposed algorithm fully exploits the semantic features of point clouds and effectively improves the accuracy of point cloud classification.

Video Decolorization Based on the CNN and LSTM Neural Network

Video decolorization is the process of transferring three-channel color videos into single-channel grayscale videos, which is essentially the decolorization operation of video frames. Most existing video decolorization algorithms directly apply image decolorization methods to decolorize video frames. However, if we only take the single-frame decolorization result into account, it will inevitably cause temporal inconsistency and flicker phenomenon meaning that the same local content between continuous video frames may display different gray values. In addition, there are often similar local content features between video frames, which indicates redundant information. To solve the preceding problems, this article proposes a novel video decolorization algorithm based on the convolutional neural network and the long short-term memory neural network. First, we design a local semantic content encoder to learn and extract the same local content of continuous video frames, which can better preserve the contrast of video frames. Second, a temporal feature controller based on the bi-directional recurrent neural networks with Long short-term memory units is employed to refine the local semantic features, which can greatly maintain temporal consistency of the video sequence to eliminate the flicker phenomenon. Finally, we take advantages of deconvolution to decode the features to produce the grayscale video sequence. Experiments have indicated that our method can better preserve the local contrast of video frames and the temporal consistency over the state of the-art.

Traditional Chinese medicine entity relation extraction based on CNN with segment attention

Extracting medical entity relations from Traditional Chinese Medicine (TCM) related article is crucial to connect domain knowledge between TCM with modern medicine. Herb accounts for the majority of Traditional Chinese Medicine, so our work mainly focuses on herb. The problem would be effectively solved by extracting herb-related entity relations from PubMed literature. In order to realize the entity relation mining, we propose a novel deep-learning model with improved layers without manual feature engineering. We design a new segment attention mechanism based on Convolutional Neural Network, which enables extracting local semantic features through word embedding. Then we classify the relations by connecting different embedding features. We first test this method on the Chemical-Induced Disease task and the experiment show better result comparing to other state-of-the-art deep learning methods. Further, we apply this method to a herbal-related data set (Herbal-Disease and Herbal Chemistry, HD-HC) constructed from PubMed to explore entity relation classification. The experiment shows superior results than other baseline methods.

Triple discriminator generative adversarial network for zero-shot image classification

One key challenge in zero-shot classification (ZSC) is the exploration of knowledge hidden in unseen classes. Generative methods such as generative adversarial networks (GANs) are typically employed to generate the visual information of unseen classes. However, the majority of these methods exploit global semantic features while neglecting the discriminative differences of local semantic features when synthesizing images, which may lead to sub-optimal results. In fact, local semantic information can provide more discriminative knowledge than global information can. To this end, this paper presents a new triple discriminator GAN for ZSC called TDGAN, which incorporates a text-reconstruction network into a dual discriminator GAN (D2GAN), allowing to realize cross-modal mapping from text descriptions to their visual representations. The text-reconstruction network focuses on key text descriptions for aligning semantic relationships to enable synthetic visual features to effectively represent images. Sharma-Mittal entropy is exploited in the loss function to make the distribution of synthetic classes be as close as possible to the distribution of real classes. The results of extensive experiments over the Caltech-UCSD Birds-2011 and North America Birds datasets demonstrate that the proposed TDGAN method consistently yields competitive performance compared to several state-of-the-art ZSC methods.