Dependency Parsing Research Articles

Distinguishing Syntactic Operations in the Brain: Dependency and Phrase-Structure Parsing.

Finding the structure of a sentence-the way its words hold together to convey meaning-is a fundamental step in language comprehension. Several brain regions, including the left inferior frontal gyrus, the left posterior superior temporal gyrus, and the left anterior temporal pole, are supposed to support this operation. The exact role of these areas is nonetheless still debated. In this paper we investigate the hypothesis that different brain regions could be sensitive to different kinds of syntactic computations. We compare the fit of phrase-structure and dependency structure descriptors to activity in brain areas using fMRI. Our results show a division between areas with regard to the type of structure computed, with the left anterior temporal pole and left inferior frontal gyrus favouring dependency structures and left posterior superior temporal gyrus favouring phrase structures.

A Cascaded Unsupervised Model for PoS Tagging

Part of speech (PoS) tagging is one of the fundamental syntactic tasks in Natural Language Processing, as it assigns a syntactic category to each word within a given sentence or context (such as noun, verb, adjective, etc.). Those syntactic categories could be used to further analyze the sentence-level syntax (e.g., dependency parsing) and thereby extract the meaning of the sentence (e.g., semantic parsing). Various methods have been proposed for learning PoS tags in an unsupervised setting without using any annotated corpora. One of the widely used methods for the tagging problem is log-linear models. Initialization of the parameters in a log-linear model is very crucial for the inference. Different initialization techniques have been used so far. In this work, we present a log-linear model for PoS tagging that uses another fully unsupervised Bayesian model to initialize the parameters of the model in a cascaded framework. Therefore, we transfer some knowledge between two different unsupervised models to leverage the PoS tagging results, where a log-linear model benefits from a Bayesian model’s expertise. We present results for Turkish as a morphologically rich language and for English as a comparably morphologically poor language in a fully unsupervised framework. The results show that our framework outperforms other unsupervised models proposed for PoS tagging.

사전학습 언어모델의 토큰 단위 문맥 표현을 이용한 한국어 의존 구문분석

사전학습 언어모델의 토큰 단위 문맥 표현을 이용한 한국어 의존 구문분석

FAR-ASS: Fact-aware reinforced abstractive sentence summarization

Automatic summarization systems provide an effective solution to today's unprecedented growth of textual data. For real-world tasks, such as data mining and information retrieval, the factual correctness of generated summary is critical. However, existing models usually focus on improving the informativeness rather than optimizing factual correctness. In this work, we present a Fact-Aware Reinforced Abstractive Sentence Summarization framework to improve the factual correctness of neural abstractive summarization models, denoted as FAR-ASS. Specifically, we develop an automatic fact extraction scheme leveraging OpenIE (Open Information Extraction) and dependency parser tools to extract structured fact tuples. Then, to quantitatively evaluate the factual correctness, we define a factual correctness score function that considers the factual accuracy and factual redundancy. We further propose to adopt reinforcement learning to improve readability and factual correctness by jointly optimizing a mixed-objective learning function. We use the English Gigaword and DUC 2004 datasets to evaluate our model. Experimental results show that compared with competitive models, our model significantly improves the factual correctness and readability of generated summaries, and also reduces duplicates while improving the informativeness.

Deep Graph-Based Character-Level Chinese Dependency Parsing

Character-level Chinese dependency parsing has been a concern of several studies that naturally handle word segmentation, POS (Part of Speech) tagging and dependency parsing jointly in an end-to-end way. Previous work mostly concentrates on a transition-based framework for this task because of its easy adaption, which is extremely important when feature representation relies heavily on the decoding strategy, particularly under the traditional statistical setting. Recently, on the one hand, sophisticated deep neural networks and deep contextualized word representations have greatly weakened the dependence between feature representation and decoding. On the other hand, (first-order) graph-based models, especially the biaffine parsers, are straightforward for dependency parsing, and meanwhile they can yield competitive parsing performance. In this paper, we make a comprehensive investigation of the deep graph-based character-level dependency parsing for Chinese. We start from an extension of a standard graph-based biaffine parser, and then exploit Chinese BERT as well as our improved encoders based on transformers to enhance the character-level dependency parsing model. We conduct a series of experiments on the Chinese benchmark datasets, showing the performances of various graph-based character-level models and analyzing the advantages of the character-level dependency parsing under the deep neural setting.

A Contemporary Ensemble Aspect-based Opinion Mining Approach for Twitter Data

Aspect-based opinion mining is one among the thought-provoking research field which focuses on the extraction of vivacious aspects from opinionated texts and polarity value associated with these. The principal aim here is to identify user sentiments about specific features of a product or service rather than overall polarity. This fine-grained polarity identification about myriad aspects of an entity is highly beneficial for individuals or business organizations. Extricating these implicit or explicit aspects can be very challenging and this paper elaborates copious aspect extraction techniques, which is decisive for aspect-based sentiment analysis. This paper presents a novel idea of combining several approaches like Part of Speech tagging, dependency parsing, word embedding, and deep learning to enrich the aspect-based sentiment analysis specially designed for Twitter data. The results show that combining deep learning with traditional techniques can produce excellent results than lexicon-based methods.

SCADA-NLI: A Natural Language Query and Control Interface for Distributed Systems

Human–Computer natural language interaction is helpful to reduce the operation and maintenance cost of the SCADA system, and it is necessary to solve the complex natural language interface problem that supports data query and real-time control. According to the complexity of natural language instructions, a hierarchical classification of semantic parsing algorithm is adopted. Firstly, the KWECS method is used to classify the intent of natural language instruction, then the TF-IDF keyword extraction algorithm combined with the cosine similarity is used to structure the key-value of the classified natural language instructions which was used into SCADA control intermediate language and then formally converted into actual control or query instruction. If the analysis fails, the complex control and query instruction analysis are carried out according to the classification results, structuring instruction parsing based on dependency parsing and SQL natural language parsing based on deep learning are adopted respectively to implement real-time control interface and database query interface. Our experimental results show that the proposed hierarchical classification of natural language comprehensive query and control interface can better solve the problem of human-computer natural language interaction in the SCADA system, and the accuracy of intent recognition reaches 96.5%. In more detail, the accuracy, precision, recall, and F-score of instruction parsing reach 88.47%, 90.21%, 89.48%, and 89.72% respectively. Especially, it provides more convenient interactive means for industrial and agricultural information management and control.

WITHDRAWN: Word Embedding Generation for Urdu Language using Word2vec model

Abstract Word embedding generation is the task of distributed representation of words on vector space. The word embeddings captures both the syntactic and semantic information to perform natural language processing tasks. The word embeddings has attained considerable attention from last few years giving their inherent applicability to Natural Language Processing like part of speech tagging, sentiment analyzing or dependency parsing etc. despite recent advancement in word embedding mappings, very little research has devoted for Urdu word embeddings where word with similar meaning will align to similar vector space and different far apart. In this research paper Word2vec model has been used for Urdu word embedding generation. The model will help to make the dense word vector representation of Urdu words which could be used for pre-trained word vectors. The results showed the proposed method can be used to improve conventional word embedding methods.

Knowledge graph mining for realty domain using dependency parsing and QAT models

The real estate business has a lot of risks, and in order to minimize them, you need a lot of information from different sources. Systems based on natural language processing can help customers find this information more easily: question answering, information retrieval, etc. The existing method of question answering requires data aligned with possible questions, which are not easy to obtain, in contrast, the knowledge-graph provides structured information. In this paper, we propose semi-automated ontology generation for the realty domain and a subsequent method for information retrieval related to the knowledge-graph of this ontology. The first contribution is the method for relation extraction method based on dependency-parsing and semantic similarity evaluation, which allows us to form ontology for a particular domain. The second contribution is knowledge-graph completion method based on question answering over text neural network. Our experimental analysis shows the efficiency of the proposed approaches.

A Rule-based Dependency Parser for Telugu: An Experiment with Simple Sentences

This paper is an attempt in building a rule-based dependency parser for Telugu which can parse simple sentences. This study adopts Pāṇini’s Grammatical (PG) tradition i.e., the dependency model to parse sentences. A detailed description of mapping semantic relations to vibhaktis (case suffixes and postpositions) in Telugu using PG is presented. The paper describes the algorithm and the linguistic knowledge employed while developing the parser. The research further provides results, which suggest that enriching the current parser with linguistic inputs can increase the accuracy and tackle ambiguity better than existing data-driven methods.

Syntactic Annotation in the I3rab Dependency Treebank

Arabic dependency parsers have a poor performance compared to parsers of other languages. Recently the impact of annotation at lexical level of dependency treebank on the overall performance of the dependency parses has been extensively investigated. This paper focuses on the impact of coarse-grained and fine-grained dependency relations on the performance of Arabic dependency parsers. Moreover, this paper introduces the annotation rules for I3rab dependency treebank. Experimentally, the obtained results showed that having an appropriate set of dependency relations improves the performance of an Arabic dependency parser up to 27.55%.

An Efficient Aspect based Sentiment Analysis Model by the Hybrid Fusion of Speech and Text Aspects

Aspect-based Sentiment Analysis (ABSA) is treated to be a challenging task in the domain of speech, as it needs the fusion of acoustic features and Linguistic features for information retrieval and decision making. The existing studies in speech are limited to speech and emotion recognition. The main objective of this work is to combine acoustic features in speech with linguistic features in text for ABSA. A deep learning and language model is implemented for acoustic feature extraction in speech. Different variants of text feature extraction techniques are used for aspect extraction in text. Trained Lexicons, Latent Dirichlet Allocation (LDA) model, Rule based approach and Efficient Named Entity Recognition (E-NER) guided dependency parsing approach has been used for aspect extraction. Sentiment with respect to the extracted aspect is analyzed using Natural Language Processing (NLP) techniques. The experimental results of the proposed model proved the effectiveness of hybrid level fusion by yielding improved results of 5.7% WER and 3% CER when compared with the traditional baseline individual linguistic and acoustic feature models.

Поиск заимствований в армянских текстах путем внутреннего стилометрического анализа

In this work we study the application of intrinsic stylometric methods to the task of plagiarism detection in Armenian texts. We use two task setups from PAN’s series of conferences on text forensics and stylometry: style change detection and style breach detection. Style change detection aims to determine whether the text is written by more than one author, while style breach detection detects the boundaries of stylistically distinct text fragments. For these tasks, we generate synthetic test sets for three genres of text: academic, literature, and news, and then use them to evaluate the effectiveness of hierarchical clustering and other relevant models from PAN conferences. We employ a standard set of character-level, lexical and readability features, and additionally perform morphological and dependency parsing of text fragments to extract syntactic features encoding author style information. The evaluation results show that the clustering-based approach fails to correctly detect style change detection in longer texts and is only marginally better for shorter texts. For style breach detection, hierarchical clustering-based approach performs better than a random baseline classifier, but the difference is not sufficient to warrant its practical use. In a complementary experiment, we show that reducing the number of features and multicollinearity in them via PCA helps to increase the precision of style breach detection methods for certain text categories.

The Application of Transformer Model Architecture for the Dependency Parsing Task

In this paper, authors discover the advantages of the attention-based neural network application to the natural language dependency-parsing task. The authors explain the architecture and show the results of comparison between attention-based neural network and long-short memory neural networks in relation to the dependency-parsing task.

Transition based neural network dependency parsing of Tibetan

In order to improve the performance of Tibetan natural language processing applications such as machine translation, sentiment analysis and other tasks, this article proposes a neural network-based method for syntactic analysis of Tibetan language dependence. Part of the corpus of Qinghai Normal University’s part-of-speech tag set is marked by the corresponding mapping relationship is transformed into the corpus annotated by the national standard part-of-speech tag set. At the same time, the CoNLL format Tibetan language dependency syntax tree library is constructed, and the method of shift-reduce plus neural network is adopted to systematically study and analyze the Tibetan language dependency syntax. Thereby improving the quality of Tibetan dependency syntactic analysis, and its accuracy rate reaches UAS:94.59%

A 3-phase approach based on sequential mining and dependency parsing for enhancing hypernym patterns performance

AbstractPatterns have been extensively used to extract hypernym relations from texts. The most popular patterns are Hearst’s patterns, formulated as regular expressions mainly based on lexical information. Experiences have reported good precision and low recall for such patterns. Thus, several approaches have been developed for improving recall. While these approaches perform better in terms of recall, it remains quite difficult to further increase recall without degrading precision. In this paper, we propose a novel 3-phase approach based on sequential pattern mining to improve pattern-based approaches in terms of both precision and recall by (i) using a rich pattern representation based on grammatical dependencies (ii) discovering new hypernym patterns, and (iii) extending hypernym patterns with anti-hypernym patterns to prune wrong extracted hypernym relations. The results obtained by performing experiments on three corpora confirm that using our approach, we are able to learn sequential patterns and combine them to outperform existing hypernym patterns in terms of precision and recall. The comparison to unsupervised distributional baselines for hypernym detection shows that, as expected, our approach yields much better performance. When compared to supervised distributional baselines for hypernym detection, our approach can be shown to be complementary and much less loosely coupled with training datasets and corpora.

Automated Features Extraction from Software Requirements Specification (SRS) Documents as The Basis of Software Product Line (SPL) Engineering

Extractive Software Product Line Engineering (SPLE) puts features on the foremost aspect in domain analysis that needs to be extracted from the existing system's artifact. Feature in SPLE, which is closely related to system functionality, has been previously studied to be extracted from source code, models, and various text documents that exist along the software development process. Source code, with its concise and normative standard, has become the most focus target for feature extraction source on many kinds of research. However, in the software engineering principle, the Software Requirements Specification (SRS) document is the basis or main reference for system functionality conformance. Meanwhile, previous researches of feature extraction from text document are conducted on a list of functional requirement sentences that have been previously prepared, not literally SRS as a whole document. So, this research proposes direct processing on the SRS document that uses requirement boilerplates for requirement sentence statement. The proposed method uses Natural Language Processing (NLP) approach on the SRS document. Sequence Part-of-Speech (POS) tagging technique is used for automatic requirement sentence identification and extraction. The features are acquired afterward from extracted requirement sentences automatically using the word dependency parsing technique. Besides, mostly the previous researches about feature extraction were using non-public available SRS document that remains classified or not accessible, so this work uses selected SRS from publicly available SRS dataset to add reproducible research value. This research proves that requirement sentence extraction directly from the SRS document is viable with precision value from 64% to 100% and recall value from 64% to 89%. While features extraction from extracted requirement sentences has success rate from 65% to 88%.

Raising the Flag: Monitoring User Perceived Disinformation on Reddit

The truth value of any new piece of information is not only investigated by media platforms, but also debated intensely on internet forums. Forum users are fighting back against misinformation, by informally flagging suspicious posts as false or misleading in their comments. We propose extracting posts informally flagged by Reddit users as a means to narrow down the list of potential instances of disinformation. To identify these flags, we built a dictionary enhanced with part of speech tags and dependency parsing to filter out specific phrases. Our rule-based approach performs similarly to machine learning models, but offers more transparency and interactivity. Posts matched by our technique are presented in a publicly accessible, daily updated, and customizable dashboard. This paper offers a descriptive analysis of which topics, venues, and time periods were linked to perceived misinformation in the first half of 2020, and compares user flagged sources with an external dataset of unreliable news websites. Using this method can help researchers understand how truth and falsehood are perceived in the subreddit communities, and to identify new false narratives before they spread through the larger population.

Dependency Parsing for Arabic Quran using Easy-First Parsing Algorithm

Arabic is the main language of Al-Quran. Nowadays, many people are studying the Language of Al-Quran, called Quran Arabic. For the beginners, it is important for them to understand the syntactic relationship in a sentence found in the Qur'an. If they do not understand enough, the interpretation will be different and wrong. It will turn into dangerous because Al-Quran is a source of guidance for Muslims’ life. Dependency parsing is very important for linguistic research, especially for rich languages such as the Arabic Language. This study aims to build dependency parsing, in order to make it easier to get to understand syntactic relationship information in sentences. This study uses a parsing method called deterministic parsing, which the method used is shift-reduce parsing with the Easy-First parsing algorithm. The evaluation used labeled attachment score calculation. The score generated from the evaluation was 69.7, beforehand, the comparison both the system results and the gold standard have been done. 62 sentences found the correct head and relation in each word. The number of words found to be wrong is not more than 3 words in one sentence. Evaluation scores produced are not exorbitant due to the complicated tagset used and lacking test sentences.

A Graph-based Model for Joint Chinese Word Segmentation and Dependency Parsing

Chinese word segmentation and dependency parsing are two fundamental tasks for Chinese natural language processing. The dependency parsing is defined at the word-level. Therefore word segmentation is the precondition of dependency parsing, which makes dependency parsing suffer from error propagation and unable to directly make use of character-level pre-trained language models (such as BERT). In this paper, we propose a graph-based model to integrate Chinese word segmentation and dependency parsing. Different from previous transition-based joint models, our proposed model is more concise, which results in fewer efforts of feature engineering. Our graph-based joint model achieves better performance than previous joint models and state-of-the-art results in both Chinese word segmentation and dependency parsing. Additionally, when BERT is combined, our model can substantially reduce the performance gap of dependency parsing between joint models and gold-segmented word-based models. Our code is publicly available at https://github.com/fastnlp/JointCwsParser