Contextual Semantic Information Research Articles

Text Summarization Method Based on Double Attention Pointer Network

A good document summary should summarize the core content of the text. Research on automatic text summarization attempts to solve this problem. The encoder-decoder model is widely used in text summarization research. Soft attention is used to obtain the required contextual semantic information during decoding. However, due to the lack of access to the key features, the generated summary deviates from the core content. In this paper, we proposed an encoder-decoder model based on a double attention pointer network (DAPT). In DAPT, the self-attention mechanism collects key information from the encoder, the soft attention and the pointer network generate more coherent core content, and the fusion of both generates accurate and coherent summaries. In addition, the improved coverage mechanism is used to address the repetition problem and improve the quality of the generated summaries. Simultaneously, scheduled sampling and reinforcement learning (RL) are combined to generate new training methods to optimize the model. Experiments on the CNN/Daily Mail dataset and the LCSTS dataset show that our model performs as well as many state-of-the-art models. The experimental analysis shows that our model achieves higher summarization performance and reduces the occurrence of repetition.

Graph Convolution Over Multiple Latent Context-Aware Graph Structures for Event Detection

Event detection is a particularly challenging problem in information extraction. The current neural network models have proved that dependency tree can better capture the correlation between candidate trigger words and related context in the sentence. However, syntactic information conveyed by the original dependency tree is insufficient for detecting trigger since the dependency tree obtained from natural language processing toolkits ignores semantic context information. Existing approaches employ a static graph structure based on original dependency tree which is incompetent in terms of distinguishing interrelations among trigger words and contextual words. So how to effectively make use of relevant information while ignoring irrelevant information from the dependency trees remains a challenging research question. To address this problem, we investigate a graph convolutional network over multiple latent context-aware graph structures to perform event detection. We exploit a multi-head attention mechanism on BERT representation and original adjacency matrix to generate multiple latent context-aware graph structures (a “dynamic cutting” strategy), which can automatically learn how to select the useful dependency information. Furthermore, we investigate graph convolutional networks with residual connections to combine the local and non-local contextual information. Experimental results on ACE2005 dataset show that our model achieves competitive performances compared with the methods based on dependency tree for event detection.

Sentiment Analysis Model Based on Self-Attention and Character-Level Embedding

Aiming at the problem of insufficient sentiment word extraction ability in existing text sentiment analysis methods and OOV (out-of-vocabulary) problem of pre-training word vectors, a neural network model combining multi-head self-attention and character-level embedding is proposed. An encoder-decoder (Seq2Seq) model is used for sentiment analysis of text. By adding character-level embedding to the word embedding layer, the OOV problem of pre-trained word vectors is solved. At the same time, the self-attention mechanism is used to perform attention calculations within the input sequence to find intra-sequence connections. The model uses bidirectional LSTM (Long Short-Term Memory) independently encodes the context semantic information of the input sequence to obtain deeper emotional features, to more effectively identify the emotional polarity of short text. The effectiveness of our method was verified on the public Twitter dataset and the movie review dataset published by ACL. Experimental results show that the accuracy of the model on the three categories of Twitter, movie reviews and IMDB datasets reaches 66.45%, 79.48% and 90.34%, respectively, which verifies that the model has excellent performance in datasets in different fields.

Chinese text classification based on attention mechanism and feature-enhanced fusion neural network

Owing to the uneven distribution of key features in Chinese texts, key features play different roles in text recognition in Chinese text classification tasks. We propose a feature-enhanced fusion model based on attention mechanism for Chinese text classification, a long short-term memory (LSTM) network, a convolutional neural network (CNN), and a feature-difference enhancement attention algorithm model. The Chinese text is digitized into a vector form containing certain semantic context information into the embedding layer to train and test the neural network by preprocessing. The feature-enhanced fusion model is implemented by double-layer LSTM and CNN modules to enhance the fusion of text features extracted from the attention mechanism for classifying the classifiers. The feature-difference enhancement attention algorithm model not only adds more weight to important text features but also strengthens the differences between them and other text features. This operation can further improves the effect of important features on Chinese text recognition. The two models are classified by the softmax function. The text classification experiments are conducted based on the Chinese text corpus. The experimental results show that compared with the contrast model, the proposed algorithm can significantly improve the recognition ability of Chinese text features.

Vehicle Logo Recognition Based on Enhanced Matching for Small Objects, Constrained Region and SSFPD Network.

Vehicle Logo Recognition (VLR) is an important part of vehicle behavior analysis and can provide supplementary information for vehicle identification, which is an essential research topic in robotic systems. However, the inaccurate extraction of vehicle logo candidate regions will affect the accuracy of logo recognition. Additionally, the existing methods have low recognition rate for most small vehicle logos and poor performance under complicated environments. A VLR method based on enhanced matching, constrained region extraction and SSFPD network is proposed in this paper to solve the aforementioned problems. A constrained region extraction method based on segmentation of the car head and car tail is proposed to accurately extract the candidate region of logo. An enhanced matching method is proposed to improve the detection performance of small objects, which augment each of training images by copy-pasting small objects many times in the unconstrained region. A single deep neural network based on a reduced ResNeXt model and Feature Pyramid Networks is proposed in this paper, which is named as Single Shot Feature Pyramid Detector (SSFPD). The SSFPD uses the reduced ResNeXt to improve classification performance of the network and retain more detailed information for small-sized vehicle logo detection. Additionally, it uses the Feature Pyramid Networks module to bring in more semantic context information to build several high-level semantic feature maps, which effectively improves recognition performance. Extensive evaluations have been made on self-collected and public vehicle logo datasets. The proposed method achieved 93.79% accuracy on the Common Vehicle Logos Dataset and 99.52% accuracy on another public dataset, respectively, outperforming the existing methods.

Contextualized CNN for Scene-Aware Depth Estimation From Single RGB Image

Directly benefited from deep learning techniques, depth estimation from single image has gained great momentum in recent years. However, most of the existing approaches treat depth prediction as an isolated problem without taking into consideration high-level semantic context information, which results in inefficient utilization of training dataset and unavoidably requires a large number of captured depth data during the training phase. To ameliorate, this paper develops a novel scene-aware contextualized convolution neural network (CCNN), which characterizes the semantic context relationship at the class-level and refines depth at the pixel-level. Our newly-proposed CCNN is built upon the intrinsic exploitation of context-dependent depth association, including inner-object continuous depth and inter-object depth change priors nearby. Specifically, rather than conducting regression on depth in single CNN, we make the first attempt to integrate both class-level and pixel-level conditional random fields (CRFs) based probabilistic graphical model into the powerful CNN framework to simultaneously learn different-level features within the same CNN layer. With our CCNN, the former model will guide the latter one to learn the contextualized RGB-Depth mapping. Hence, CCNN has desirable properties in both class-level integrity and pixel-level discrimination, which makes it ideal to share such two-level convolutional features in parallel during the end-to-end training with the commonly-used back-propagation algorithm. We conduct extensive experiments and comprehensive evaluations on public benchmarks involving various indoor and outdoor scenes, and all the experiments confirm that, our method outperforms the state-of-the-art depth estimation methods, especially for the cases where only small-scale training data are readily available.

Multi-Scale Remote Sensing Semantic Analysis Based on a Global Perspective

Remote sensing image captioning involves remote sensing objects and their spatial relationships. However, it is still difficult to determine the spatial extent of a remote sensing object and the size of a sample patch. If the patch size is too large, it will include too many remote sensing objects and their complex spatial relationships. This will increase the computational burden of the image captioning network and reduce its precision. If the patch size is too small, it often fails to provide enough environmental and contextual information, which makes the remote sensing object difficult to describe. To address this problem, we propose a multi-scale semantic long short-term memory network (MS-LSTM). The remote sensing images are paired into image patches with different spatial scales. First, the large-scale patches have larger sizes. We use a Visual Geometry Group (VGG) network to extract the features from the large-scale patches and input them into the improved MS-LSTM network as the semantic information, which provides a larger receptive field and more contextual semantic information for small-scale image caption so as to play the role of global perspective, thereby enabling the accurate identification of small-scale samples with the same features. Second, a small-scale patch is used to highlight remote sensing objects and simplify their spatial relations. In addition, the multi-receptive field provides perspectives from local to global. The experimental results demonstrated that compared with the original long short-term memory network (LSTM), the MS-LSTM’s Bilingual Evaluation Understudy (BLEU) has been increased by 5.6% to 0.859, thereby reflecting that the MS-LSTM has a more comprehensive receptive field, which provides more abundant semantic information and enhances the remote sensing image captions.

The user preference identification for product improvement based on online comment patch

Online comments have become a valuable source for designers for the purpose of product improvement. However, the implicitly expressed users’ preferences on multiple product attributes in incomplete online comments make it difficult to extract useful information to improve the product from the online comments. In order to identify the users’ preference from the perspective of product improvement, the comment extension mining model is proposed to patch up the online reviews based on the semantic similarity and emotional resemblance. First of all, for the sake of mining the full context semantic information of the keywords in comments, they are transferred into vectors by the word2vec method. Next, smart semantic distance measurement models are developed to match the online comments with the standard comment templates that express users’ sentiment on product attributes based on the semantic similarity. Moreover, the fine-grained matching neural networks are designed to further match the reviews of each product attribute to its standard sentiment templates according to the user’s overall emotions towards the product. Finally, the KANO model is introduced to depict users’ preferences and develop product improvement strategies. The experiment on laptop product confirms that our method is effective.

An Entity Linking Method Based on Entity Category and Word Embedding

Named entity linking is a process of linking a given reference in a document to a knowledge base. In natural language processing, entity linking can enhance the computer’s understanding of unstructured text data. Applying traditional entity linking methods, especially entity linking methods for person names and organization names, has its limitations. Similar vocabulary as an entity to be linked is difficult to make full use of its contextual semantic information for ambiguity elimination. This paper makes full use of the entity’s category attribute and the semantic information contained in the context to design an entity linking method based on entity category and semantic word embedding. First, Training text classification model based on corpus to obtain entity attributes. Then the semantic feature is extracted by the word vector template to perform entity disambiguation through the semantic classification model. Finally, the results of the entity linking are predicted by means of model ensemble. Experiments show that the accuracy of the method after fusion on the entity linking dataset has improved.

Human Detection Aided by Deeply Learned Semantic Masks

Human detection is one of the long-standing computer vision tasks, and it has been a cornerstone for many real-world applications, such as photo album organization, video surveillance, and autonomous driving. Benefiting from deep learning technologies, such as convolutional neural networks and modern object detectors, have been achieving much improved accuracy in generic object detection tasks. In this paper, we aim to improve deep learning-based human detection. Our main idea is to exploit semantic context information for human detection by using deep-learnt semantic features provided by semantic segmentation masks. Segmentation masks play as an attention mechanism and enforce the detectors to focus on the image regions where potential object candidates are likely to appear. Meanwhile, the extra segmentation mask channel can also guide the convolutional kernels to automatically learn more discriminative features which make it easier to distinguish the background and foreground. We implement our methods with two popular detection frameworks, i.e., faster R-CNN and SSD and experimentally analyze the effectiveness of the proposed methods. Evaluation results on the widely used MS-COCO dataset and the very recent CrowdHuman dataset are provided. Our proposed methods outperform the baseline detectors and achieve better performance on highly occluded human detection.

Variational Context: Exploiting Visual and Textual Context for Grounding Referring Expressions.

We focus on grounding (i.e., localizing or linking) referring expressions in images, e.g., "largest elephant standing behind baby elephant". This is a general yet challenging vision-language task since it does not only require the localization of objects, but also the multimodal comprehension of context - visual attributes (e.g., "largest", "baby") and relationships (e.g., "behind") that help to distinguish the referent from other objects, especially those of the same category. Due to the exponential complexity involved in modeling the context associated with multiple image regions, existing work oversimplifies this task to pairwise region modeling by multiple instance learning. In this paper, we propose a variational Bayesian method, called Variational Context, to solve the problem of complex context modeling in referring expression grounding. Specifically, our framework exploits the reciprocal relation between the referent and context, i.e., either of them influences estimation of the posterior distribution of the other, and thereby the search space of context can be greatly reduced. In addition to reciprocity, our framework considers the semantic information of context, i.e., the referring expression can be reproduced based on the estimated context. We also extend the model to unsupervised setting where no annotation for the referent is available. Extensive experiments on various benchmarks show consistent improvement over state-of-the-art methods in both supervised and unsupervised settings.

Sparse attention based separable dilated convolutional neural network for targeted sentiment analysis

Long short-term memory networks (LSTM) and classical convolutional neural networks (CNN) are two critical methods for the task of targeted sentiment analysis, but LSTM are difficult to parallelize and time-inefficient, and classical CNN can only capture local semantic features. To this end, this paper first proposes a sparse attention based separable dilated convolutional neural network (SA-SDCCN), which consists of multichannel embedding layer, separable dilated convolution module, sparse attention layer, and output layer. Specifically, our work is mainly concentrated on the first three parts. In multichannel embedding layer, semantic and sentiment embeddings are incorporated into an embedding tensor, which builds richer representations over the input sequence. In separable dilated convolution module, long-range contextual semantic information is explored and multi-scale contextual semantic dependencies are aggregated simultaneously through diverse dilation rates. Moreover, the separable structure further reduces the model parameters. In sparse attention layer, sentiment-oriented components are noticed according to the features of specific target entity. Finally, some experiments on three benchmark datasets demonstrate that SA-SDCCN achieves comparable or even better performance than state-of-the-art methods in terms of higher parallelism and lower computational cost.

Separable reverse connected network for efficient multi-scale vehicle detection

Vehicle detection is involved in a wide range of intelligent transportation and smart city applications, and the demand of fast and accurate detection of vehicles is increasing. In this article, we propose a convolutional neural network-based framework, called separable reverse connected network, for multi-scale vehicles detection. In this network, reverse connected structure enriches the semantic context information of previous layers, while separable convolution is introduced for sparse representation of heavy feature maps generated from subnetworks. Further, we use multi-scale training scheme, online hard example mining, and model compression technique to accelerate the training process as well as reduce the parameters. Experimental results on Pascal Visual Object Classes (VOC) 2007 + 2012 and MicroSoft Common Objects in COntext (MS COCO) 2014 demonstrate the proposed method yields state-of-the-art performance. Moreover, by separable convolution and model compression, the network of two-stage detector is accelerated by about two times with little loss of detection accuracy.

Dialogue intent classification with character-CNN-BGRU networks

Dialogue intent classification plays a significant role in human-computer interaction systems. In this paper, we present a hybrid convolutional neural network and bidirectional gated recurrent unit neural network (CNN-BGRU) architecture to classify the intent of a dialogue utterance. First, character embeddings are trained and used as the inputs of the proposed model. Second, a CNN is used to extract local features from each utterance, and a maximum pooling layer is applied to select the most crucial latent semantic factors. A bidirectional gated recurrent unit (BGRU) layer architecture is used to capture the contextual semantic information. Then, two feature maps, which are the outputs of the two architectures, are integrated into the final utterance representation. The proposed model can utilize local semantic and contextual information to recognize and classify the user dialogue intent in an efficient way. The proposed model is evaluated based on a social media processing (SMP) data set and a real conversational data set. The experimental results show that the proposed model outperforms the corresponding traditional methods. In addition, compared to the CNN and BGRU methods, the classification accuracy of the proposed model is 1.4% higher for the SMP data set.

Region search based on hybrid convolutional neural network in optical remote sensing images

Currently, big data is a new and hot issue. Particularly, the rapid growth of the Internet of Things causes a sharp growth of data. Enormous amounts of networking sensors are continuously collecting and transmitting data to be stored and processed in the cloud, including remote sensing data, environmental data, and geographical data. And region is regarded as the very important object in remote sensing data, which is mainly researched in this article. Region search is a crucial task in remote sensing process, especially for military area and civilian fields. It is difficult to fast search region accurately and achieve generalizability of the regions’ features due to the complex background information, as well as the smaller size. Especially, when processing region search in large-scale remote sensing image, detailed information as the feature can be extracted in inner region. To overcome the above difficulty region search task, we propose an accurate and fast region search in optical remote sensing images under cloud computing environment, which is based on hybrid convolutional neural network. The proposed region search method partitioned into four processes. First, fully convolutional network is adopted to produce all the candidate regions that contain the possible object regions. This process avoids exhaustive search for input images. Then, the features of all candidate regions are extracted by a fast region-based convolutional neural network structure. Third, we design a new difficult sample mining method for the training process. At the end, in order to improve the region search precision, we use an iterative bounding box regression algorithm to normalize the detected bounding boxes, in which the regions contain candidate objects. The proposed algorithm is evaluated on optical remote sensing images acquired from Google Earth. Finally, we conduct the experiments, and the obtained results show that the proposed region search method constantly achieves better results regardless of the type of images tested. Compared with traditional region search methods, such as region-based convolutional neural network and newest feature extraction frameworks, our proposed methods show better robustness with complex context semantic information and backgrounds.

Improving User Attribute Classification with Text and Social Network Attention

User attribute classification is an important research topic in social media user profiling, which has great commercial value in modern advertisement systems. Existing research on user profiling has mostly focused on manually handcrafted features for different attribute classification tasks. However, these research has partly overlooked the social relation of users. We propose an end-to-end neural network model called the social convolution attention neural network. Our model leverages the convolution attention mechanism to automatically extract user features with respect to different attributes from social texts. The proposed model can capture the social relation of users by combining semantic context and social network information, and improve the performance of attribute classification. We evaluate our model in the gender, age, and geography classification tasks based on the dataset from SMP CUP 2016 competition, respectively. The experimental results demonstrate that the proposed model is effective in automatic user attribute classification with a particular focus on fine-grained user information. We propose an effective model based on the convolution attention mechanism and social relation information for user attribute classification. The model can significantly improve the accuracy in various user attribute classification tasks.

Attention-based Autoencoder Topic Model for Short Texts

Due to sparse word co-occurrences, traditional topic models work poorly on short texts. Recent work aggregates short texts to augment word co-occurrence. However, aggregation has a constraint on topic distribution over a document. Thus, we propose an Attention-based Autoencoder Topic Model (AATM) in this paper. The attention mechanism of AATM emphasizes relevant information and improves topic coherence. We also incorporate externally well-trained word embeddings to introduce contextual semantic information. Moreover, AATM consists of a phrase model and a knowledge-based ranking model to discover phrase-level topics and inter-topic rankings, respectively. Experimental results on real-world data validate the efficacy of AATM.

AELA-DLSTMs: Attention-Enabled and Location-Aware Double LSTMs for aspect-level sentiment classification

Aspect-level sentiment classification, as a fine-grained task in sentiment classification, aiming to extract sentiment polarity from opinions towards a specific aspect word, has been made tremendous improvements in recent years. There are three key factors for aspect-level sentiment classification: contextual semantic information towards aspect words, correlations between aspect words and their context words, and location information of context words with regard to aspect words. In this paper, two models named AE-DLSTMs (Attention-Enabled Double LSTMs) and AELA-DLSTMs (Attention-Enabled and Location-Aware Double LSTMs) are proposed for aspect-level sentiment classification. AE-DLSTMs takes full advantage of the DLSTMs (Double LSTMs) which can capture the contextual semantic information in both forward and backward directions towards aspect words. Meanwhile, a novel attention weights generating method that combines aspect words with their contextual semantic information is designed so that those weights can make better use of the correlations between aspect words and their context words. Besides, we observe that context words with different distances or different directions towards aspect words have different contributions in sentiment polarity. Based on AE-DLSTMs, the location information of context words by assigning different weights is incorporated in AELA-DLSTMs to improve the accuracy. Experiments are conducted on two English datasets and one Chinese dataset. The experimental results have confirmed that our models can make remarkable improvements and outperform all the baseline models in all datasets, improving the accuracy of 1.67 percent to 4.77 percent in different datasets compared with baseline models.11Our code is open-source and available at https://github.com/rxt2012kc/AELA-DLSTMs.

Universal neural basis of structure building evidenced by network modulations emerging from Broca's area: The case of Chinese.

The basic steps in building up language involve binding words of different categories into a hierarchical structure. To what extent these steps are universal or differ across languages is an open issue. Here we examine the neural dynamics of phrase structure building in Chinese-a language that in contrast to other languages heavily depends on contextual semantic information. We used functional magnetic resonance imaging and dynamic causal modeling to identify the relevant brain regions and their dynamic relations. Language stimuli consisted of syntax-driving determiners, semantics-embedded classifiers, and nonverbal symbols making up for two-component sequences manipulated by the factors structure (phrase/list) and number of words (2-word/1-word). Processing phrases compared with word lists elicited greater activation in the anterior part of Broca's area, Brodmann area (BA) 45, and the left posterior superior/middle temporal gyri (pSTG/pMTG), while processing two words against one word led to stronger involvement of the left BA 45, BA 44, and insula. Differential network modulations emerging from subparts of Broca's area revealed that phrasal construction in particular highly modulated the direct connection from BA 44 to left pMTG, suggesting BA 44's primary role in phrase structure building. Conversely, the involvement of BA 45 rather appears sensitive to the reliance on lexico-semantic information in Chinese. Against the background of previous findings from other languages, the present results indicate that phrase structure building has a universal neural basis within the left fronto-temporal network. Most importantly, they provide the first evidence demonstrating that the structure-building network may be modulated by language-specific characteristics.

OmniArt

Baselines are the starting point of any quantitative multimedia research, and benchmarks are essential for pushing those baselines further. In this article, we present baselines for the artistic domain with a new benchmark dataset featuring over 2 million images with rich structured metadata dubbed OmniArt. OmniArt contains annotations for dozens of attribute types and features semantic context information through concepts, IconClass labels, color information, and (limited) object-level bounding boxes. For our dataset we establish and present baseline scores on multiple tasks such as artist attribution, creation period estimation, type, style, and school prediction. In addition to our metadata related experiments, we explore the color spaces of art through different types and evaluate a transfer learning object recognition pipeline.