Semantic Representation Of Sentence Research Articles

A deep learning approach to recognizing fine-grained expressway location reference from unstructured texts in Chinese

Complex nested and discontinuous location references are common in unstructured text. Extracting them is essential for accurate location information retrieval and spatial inference. However, traditional methods struggle with these references due to annotation system and model architecture limitations. In this study, we introduce a deep learning approach to uniformly recognize flat, nested, and discontinuous location references, motivated by recognizing fine-grained expressway location references. The approach uses a pre-trained language model to generate semantic sentence representations and a distance and direction-aware Transformer for contextual encoding. Then, it recognizes location references by modeling the adjacency and boundary relations between word pairs. We evaluated the approach on seven benchmark datasets and compared it with state-of-the-art methods. The results show that the approach achieves higher accuracy with faster inference, validating our modeling paradigm and architecture. The ablation study further confirms the effectiveness of submodules in architecture. These findings can provide valuable insights for developing advanced unified location reference recognition methods. Moreover, the detailed labeled dataset for location references can facilitate the evaluation and comparison of unified recognition methods and systems.

Attention-based Stacked Bidirectional Long Short-term Memory Model for Word Sense Disambiguation

Word sense disambiguation is a basic task in Natural Language Processing which aims to identify the most appropriate sense of ambiguous words in different contexts by applying algorithm models. In this work, we propose a model that uses a stacked bidirectional Long Short-Term Memory neural network and attention mechanism to determine the sense of ambiguous words. First, the stacked bidirectional Long Short-Term Memory is employed for deep embedding-based representation of sentences containing ambiguous words. Then, we utilize the self-attention mechanism to highlight the contextual features of ambiguous words, and then construct the overall semantic representation of sentences. Finally, the sentence semantic representation is applied to the multilayer perception classifier to generate the appropriate category of the ambiguous word sense items. This model is tested on the Semeval-2007 task-17: English lexical samples dataset and using examples of ambiguous words sourced from Oxford, Cambridge, and Collins dictionaries as extra test datasets. The effectiveness of the proposed approach is demonstrated via comparison with existing word sense disambiguation approaches. Our experimental results show that the proposed model outperforms other word sense disambiguation methods in terms of the evaluation metrics (Average Accuracy, Micro F1-Score, Kappa, and Matthews Correlation Coefficient), and exhibits strong interpretability.

The Complexity of Semantic Components in Linguistic Era

In Section One, a brief account is made to describe how a semantic theory explains and shows the interpretative ability of speakers. Any lexical item in a given sentence is supplied with more senses, by a dictionary, than it can bear. The task of the projection rules is to select the most suitable sense of each item in that sentence. The semantic components discussed according to the phonological and semantic representation of sentence which is the basic concern of Section Two, are associated with a linguistic description of a natural language proposed the specification of the interconnections between the three components: syntax, phonology and semantics to be of mutual dependency. Section Three advocates another split from the Standard Theory which is called Generative Semantics: a theory developed by Lackoff, McCawley, Ross and others has found its way since 1968 as a strong opponent to the Extended Theory. We also, in Section Four discuss the semantic acceptability in linguistics through drawing a distinction between grammaticality (a property belonging to a competence) and acceptability (a property identical in performance).

Densely Enhanced Semantic Network for Conversation System in Social Media

The human–computer conversation system is a significant application in the field of multimedia. To select an appropriate response, retrieval-based systems model the matching between the dialogue history and response candidates. However, most of the existing methods cannot fully capture and utilize varied matching patterns, which may degrade the performance of the systems. To address the issue, a densely enhanced semantic network (DESN) is proposed in our work. Given a multi-turn dialogue history and a response candidate, DESN first constructs the semantic representations of sentences from the word perspective, the sentence perspective, and the dialogue perspective. In particular, the dialogue perspective is a novel one introduced in our work. The dependencies between a single sentence and the whole dialogue are modeled from the dialogue perspective. Then, the response candidate and each utterance in the dialogue history are made to interact with each other. The varied matching patterns are captured for each utterance–response pair by using a dense matching module. The matching patterns of all the utterance–response pairs are accumulated in chronological order to calculate the matching degree between the dialogue history and the response. The responses in the candidate pool are ranked with the matching degree, thereby returning the most appropriate candidate. Our model is evaluated on the benchmark datasets. The experimental results prove that our model achieves significant and consistent improvement when compared with other baselines.

TreeLSTM with tag-aware hypernetwork for sentence representation

Tree-structured neural networks, such as TreeLSTM and its variants, have proven effective for learning semantic representations of sentences, which are useful for a variety of tasks in natural language processing such as text categorisation, text semantic matching and machine translation. These neural network models take as inputs parse trees of sentences, which are generated by a language parser. However, most existing tree-structured neural network models lack the ability of distinguishing different syntactic compositions, thus the expressive power of these models is limited. Moreover, the syntactic knowledge provided by Part-of-Speech tags in a parse tree has not been fully utilised in existing tree-structured neural network models. It is expected that such syntactic knowledge should help distinguish syntactic compositions, so should result in better semantic representation. This paper proposes a novel neural network model, TagHyperTreeLSTM, which contains two components, a tag-aware hypernetwork and a sentence encoder. The tag-aware hypernetwork, which accepts tags as inputs, generates the parameters of the sentence encoder dynamically in order to distinguish different syntactic compositions. The sentence encoder, which accepts words as inputs, generates the final sentence representation. Experimental results show that the proposed model achieves superior or competitive performance in text classification and text semantic matching based on six benchmark datasets when compared against previous tree-structured models.

A Deep Paraphrase Identification Model Interacting Semantics with Syntax

Paraphrase identification is central to many natural language applications. Based on the insight that a successful paraphrase identification model needs to adequately capture the semantics of the language objects as well as their interactions, we present a deep paraphrase identification model interacting semantics with syntax (DPIM-ISS) for paraphrase identification. DPIM-ISS introduces the linguistic features manifested in syntactic features to produce more explicit structures and encodes the semantic representation of sentence on different syntactic structures by means of interacting semantics with syntax. Then, DPIM-ISS learns the paraphrase pattern from this representation interacting the semantics with syntax by exploiting a convolutional neural network with convolution-pooling structure. Experiments are conducted on the corpus of Microsoft Research Paraphrase (MSRP), PAN 2010 corpus, and PAN 2012 corpus for paraphrase plagiarism detection. The experimental results demonstrate that DPIM-ISS outperforms the classical word-matching approaches, the syntax-similarity approaches, the convolution neural network-based models, and some deep paraphrase identification models.

Learning semantic sentence representations from visually grounded language without lexical knowledge

AbstractCurrent approaches to learning semantic representations of sentences often use prior word-level knowledge. The current study aims to leverage visual information in order to capture sentence level semantics without the need for word embeddings. We use a multimodal sentence encoder trained on a corpus of images with matching text captions to produce visually grounded sentence embeddings. Deep Neural Networks are trained to map the two modalities to a common embedding space such that for an image the corresponding caption can be retrieved and vice versa. We show that our model achieves results comparable to the current state of the art on two popular image-caption retrieval benchmark datasets: Microsoft Common Objects in Context (MSCOCO) and Flickr8k. We evaluate the semantic content of the resulting sentence embeddings using the data from the Semantic Textual Similarity (STS) benchmark task and show that the multimodal embeddings correlate well with human semantic similarity judgements. The system achieves state-of-the-art results on several of these benchmarks, which shows that a system trained solely on multimodal data, without assuming any word representations, is able to capture sentence level semantics. Importantly, this result shows that we do not need prior knowledge of lexical level semantics in order to model sentence level semantics. These findings demonstrate the importance of visual information in semantics.

Apps-based Machine Translation on Smart Media Devices - A Review

Machine Translation Systems are part of Natural Language Processing (NLP) that makes communication possible among people using their own native language through computer and smart media devices. This paper describes recent progress in language dictionaries and machine translation commonly used for communications and social interaction among people or Internet users worldwide who speak different languages. Problems of accuracy and quality related to computer translation systems encountered in web & Apps-based translation are described and discussed. Possible programming solutions to the problems are also put forward to create software tools that are able to analyze and synthesize language intelligently based on semantic representation of sentences and phrases. Challenges and problems on Apps-based machine translation on smart devices towards AI, NLP, smart learning and understanding still remain until now, and need to be addressed and solved through collaboration between computational linguists and computer scientists.

Multi-Turn Response Selection for Chatbots With Hierarchical Aggregation Network of Multi-Representation

Matching an appropriate response with its multi-turn context is a crucial challenge in retrieval-based chatbots. Current studies construct multiple representations of context and response to facilitate response selection, but they use these representations in isolation and ignore the relationships among representations. To address these problems, we propose a hierarchical aggregation network of multi-representation (HAMR) to leverage abundant representations sufficiently and enhance valuable information. First, we employ bidirectional recurrent neural networks (BiRNN) to extract syntactic and semantic representations of sentences and use a self-aggregation mechanism to combine these representations. Second, we design a matching aggregation mechanism for fusing different matching information between each utterance in context and response, which is generated by an attention mechanism. By considering the candidate response as the real part of the context, we try to integrate all of them in chronological order and then accumulate the vectors to calculate the final matching degree. An extensive empirical study on two multi-turn response selection data sets indicates that our proposed model achieves a new state-of-the-art result.

Leveraging syntactic and semantic graph kernels to extract pharmacokinetic drug drug interactions from biomedical literature.

BackgroundInformation about drug–drug interactions (DDIs) supported by scientific evidence is crucial for establishing computational knowledge bases for applications like pharmacovigilance. Since new reports of DDIs are rapidly accumulating in the scientific literature, text-mining techniques for automatic DDI extraction are critical. We propose a novel approach for automated pharmacokinetic (PK) DDI detection that incorporates syntactic and semantic information into graph kernels, to address the problem of sparseness associated with syntactic-structural approaches. First, we used a novel all-path graph kernel using shallow semantic representation of sentences. Next, we statistically integrated fine-granular semantic classes into the dependency and shallow semantic graphs.ResultsWhen evaluated on the PK DDI corpus, our approach significantly outperformed the original all-path graph kernel that is based on dependency structure. Our system that combined dependency graph kernel with semantic classes achieved the best F-scores of 81.94 % for in vivo PK DDIs and 69.34 % for in vitro PK DDIs, respectively. Further, combining shallow semantic graph kernel with semantic classes achieved the highest precisions of 84.88 % for in vivo PK DDIs and 74.83 % for in vitro PK DDIs, respectively.ConclusionsWe presented a graph kernel based approach to combine syntactic and semantic information for extracting pharmacokinetic DDIs from Biomedical Literature. Experimental results showed that our proposed approach could extract PK DDIs from literature effectively, which significantly enhanced the performance of the original all-path graph kernel based on dependency structure.

Deep Sentence Embedding Using Long Short-Term Memory Networks: Analysis and Application to Information Retrieval

This paper develops a model that addresses sentence embedding, a hot topic in current natural language processing research, using recurrent neural networks with Long Short-Term Memory (LSTM) cells. Due to its ability to capture long term memory, the LSTM-RNN accumulates increasingly richer information as it goes through the sentence, and when it reaches the last word, the hidden layer of the network provides a semantic representation of the whole sentence. In this paper, the LSTM-RNN is trained in a weakly supervised manner on user click-through data logged by a commercial web search engine. Visualization and analysis are performed to understand how the embedding process works. The model is found to automatically attenuate the unimportant words and detects the salient keywords in the sentence. Furthermore, these detected keywords are found to automatically activate different cells of the LSTM-RNN, where words belonging to a similar topic activate the same cell. As a semantic representation of the sentence, the embedding vector can be used in many different applications. These automatic keyword detection and topic allocation abilities enabled by the LSTM-RNN allow the network to perform document retrieval, a difficult language processing task, where the similarity between the query and documents can be measured by the distance between their corresponding sentence embedding vectors computed by the LSTM-RNN. On a web search task, the LSTM-RNN embedding is shown to significantly outperform several existing state of the art methods. We emphasize that the proposed model generates sentence embedding vectors that are specially useful for web document retrieval tasks. A comparison with a well known general sentence embedding method, the Paragraph Vector, is performed. The results show that the proposed method in this paper significantly outperforms it for web document retrieval task.

Effect of the Compatibility between the Noun Phrase and the Verb in Forming Semantic Representations of Sentences

Effect of the Compatibility between the Noun Phrase and the Verb in Forming Semantic Representations of Sentences

Language

INTRODUCTION: The study focuses on how cortical activations are modulated by the processing of different linguistic units when people perform semantic language tasks. From the perspective of cognitive science, semantic activation entails a process of combining the meanings of small parts (units) into a large whole (4). Brain mapping data indicated a significant difference in semantic retrieval of Chinese single characters and phrases (1, 3). Here we hypothesize that distinct brain areas are recruited in processing different linguistic units semantically (2). We used sentences and characters in this study to verify this hypothesis. METHODS: Nine native Chinese speakers participated. There were two experimental tasks, character semantic judgment and sentential semantic judgment. Each task was contrasted with its own baseline which controlled for the influence of processing efforts. The study was performed on a 2 T GE/Elscint Prestige whole-body scanner. A T2*-weighted gradient-echo EPI sequence was used, with the slice thickness = 6 mm, in-plane resolution = 2.9mm x 2.9mm, and TR/TE/FA= 2000 ms/45 ms/90 degree. Eighteen contiguous axial slices were acquired to cover the whole brain. SPM99 was used for image analysis. RESULTS AND CONCLUSION: Areas common to semantic processing of characters and sentences included left infeo-frontal gyrus (BA 45 and 47). Left middle temporal and precuneus gyri (BA 19, 39, 37) contributed to semantic representation of sentences, but not characters. Other areas mediating the semantic processing of sentences were cuneus, lingual gyrus, pre-and post-central gyri, supero-medial frontal cortex, and cingulate. Our fMRI results showed that while left inferior frontal cortex is responsible for activation of semantic valence of language stimuli in general, meaning processing of large linguistic units recruits cortical areas that are not active in processing simple language units.

An approach to natural-language semantics in logic programming

The author presents the main lines of an approach to the semantics of natural-language sentences within the logic programming framework. A formalism and tools are defined to represent noun phrases, verb phrases, adjectives, determiners, and some adverbs with a degree of precision deemed suitable for a natural-language front end. The formalism used is based on first-order logic augmented by PROLOG calls. A methodology is presented rather than an exhaustive list of results. The author also gives the main lines of an approach to building the semantic representation of sentences, i.e., by the application of rewrite rules associated with contextual preconditions of application.