Shallow Parsing Research Articles

Multi Task Learning Based Shallow Parsing for Indian Languages

Shallow Parsing is an important step for many Natural Language Processing tasks. Although shallow parsing has a rich history for resource rich languages, it is not the case for most Indian languages. Shallow Parsing consists of POS Tagging and Chunking. Our study focuses on developing shallow parsers for Indian languages. As part of shallow parsing we included morph analysis as well. For the study, we first consolidated available shallow parsing corpora for 7 Indian Languages (Hindi, Kannada, Bangla, Malayalam, Marathi, Urdu, Telugu) for which treebanks are publicly available. We then trained models to achieve state of the art performance for shallow parsing in these languages for multiple domains. Since analyzing the performance of model predictions at sentence level is more realistic, we report the performance of these shallow parsers not only at the token level, but also at the sentence level. We also present machine learning techniques for multitask shallow parsing. Our experiments show that fine-tuned contextual embedding with multi-task learning improves the performance of multiple as well as individual shallow parsing tasks across different domains. We show the transfer learning capability of these models by creating shallow parsers (only with POS and Chunk) for Gujarati, Odia, and Punjabi for which no treebanks are available. As a part of this work, we will be releasing the Indian Languages Shallow Linguistic (ILSL) benchmarks for 10 Indian languages including both the major language families Indo-Aryan and Dravidian as common building blocks that can be used to evaluate and understand various linguistic phenomena found in Indian languages and how well newer approaches can tackle them.

Identification of monolingual and code-switch information from English-Kannada code-switch data

<span lang="EN-US">Code-switching is a very common occurrence in social media communication, predominantly found in multilingual countries like India. Using more than one language in communication is known as code-switching or code-mixing. Some of the important applications of code-switch are machine translation (MT), shallow parsing, dialog systems, and semantic parsing. Identifying code-switch and monolingual information is useful for better communication in online networking websites. In this paper, we performed a character level n-gram approach to identify monolingual and code-switch information from English-Kannada social media data. We paralleled various machine learning techniques such as naïve Bayes (NB), support vector classifier (SVC), logistic regression (LR) and neural network (NN) on English-Kannada code-switch (EKCS) data. From the proposed approach, it is observed that the character level n-gram approach provides 1.8% to 4.1% of improvement in terms of Accuracy and 1.6% to 3.8% of improvement in F1-score. Also observed that SVC and NN techniques are outperformed in terms of accuracy (97.9%) and F1-score (98%) with character level n-gram.</span>

Conserving Semantic Unit Information and Simplifying Syntactic Constituents to Improve Implicit Discourse Relation Recognition.

Implicit discourse relation recognition (IDRR) has long been considered a challenging problem in shallow discourse parsing. The absence of connectives makes such relations implicit and requires much more effort to understand the semantics of the text. Thus, it is important to preserve the semantic completeness before any attempt to predict the discourse relation. However, word level embedding, widely used in existing works, may lead to a loss of semantics by splitting some phrases that should be treated as complete semantic units. In this article, we proposed three methods to segment a sentence into complete semantic units: a corpus-based method to serve as the baseline, a constituent parsing tree-based method, and a dependency parsing tree-based method to provide a more flexible and automatic way to divide the sentence. The segmented sentence will then be embedded at the level of semantic units so the embeddings could be fed into the IDRR networks and play the same role as word embeddings. We implemented our methods into one of the recent IDRR models to compare the performance with the original version using word level embeddings. Results show that proper embedding level better conserves the semantic information in the sentence and helps to enhance the performance of IDRR models.

Deep Learning-based Sequence Labeling Tools for Nepali

A Part-of-Speech (POS) tagger and Chunker (or shallow parser) are sequence labeling tools, crucial for improving the accuracy of Natural Language Processing (NLP) tasks like parsing, named entity recognition, sentiment analysis, information extraction, and so on. Developing such tools for a low-resource language is an arduous task. Nepali is a relatively resource-poor Indian language and has not been able to evolve from a computational perspective. Therefore, we present effective part-of-speech tagging and chunking tools for the Nepali text using sequential deep learning models—Bidirectional Long Short-Term Memory Network with a Conditional Random Field Layer (BI-LSTM-CRF) and other LSTM-based models exploring both character and word embeddings of the Nepali texts. Word Embedding has been used to capture syntactic as well as semantic information whereas character embedding has been applied to capture the morphological as well as shape information of words and also to handle the out-of-vocabulary problem. The developed chunker is the first statistical chunker for the Nepali language. A baseline model with a Conditional Random Field has also been developed to identify the optimum feature set for the aforementioned tasks. The BI-LSTM-CRF model produced an accuracy of 99.20% and 98.40%, for Nepali POS tagging and chunking, respectively. This is the highest-ever accuracy for Nepali. Thorough error analysis and observations have also been reported with examples. The developed tools can help advance research in Nepali language processing, improve the accuracy of language technology applications, and contribute to the preservation and promotion of the Nepali language.

Toward a shallow discourse parser for Turkish

Abstract One of the most interesting aspects of natural language is how texts cohere, which involves the pragmatic or semantic relations that hold between clauses (addition, cause-effect, conditional, similarity), referred to as discourse relations. A focus on the identification and classification of discourse relations appears as an imperative challenge to be resolved to support tasks such as text summarization, dialogue systems, and machine translation that need information above the clause level. Despite the recent interest in discourse relations in well-known languages such as English, data and experiments are still needed for typologically different and less-resourced languages. We report the most comprehensive investigation of shallow discourse parsing in Turkish, focusing on two main sub-tasks: identification of discourse relation realization types and the sense classification of explicit and implicit relations. The work is based on the approach of fine-tuning a pre-trained language model (BERT) as an encoder and classifying the encoded data with neural network-based classifiers. We firstly identify the discourse relation realization type that holds in a given text, if there is any. Then, we move on to the sense classification of the identified explicit and implicit relations. In addition to in-domain experiments on a held-out test set from the Turkish Discourse Bank (TDB 1.2), we also report the out-domain performance of our models in order to evaluate its generalization abilities, using the Turkish part of the TED Multilingual Discourse Bank. Finally, we explore the effect of multilingual data aggregation on the classification of relation realization type through a cross-lingual experiment. The results suggest that our models perform relatively well despite the limited size of the TDB 1.2 and that there are language-specific aspects of detecting the types of discourse relation realization. We believe that the findings are important both in providing insights regarding the performance of the modern language models in a typologically different language and in the low-resource scenario, given that the TDB 1.2 is 1/20th of the Penn Discourse TreeBank in terms of the number of total relations.

Indonesian news classification application with named entity recognition approach

Nowadays, many netizens search for news via search engines with countless amounts of information, so it is increasingly difficult to determine when the number of news articles that appear changes very quickly and dynamically. Thus, it is necessary to process the extraction of news information to display the core information of the news. Problems arise, especially in Indonesian, which has a structure of various noun phrase entities with shallow parsing or grammatical induction. Named Entity Recognition (NER) has the opportunity to overcome this because it can extract news entities in depth, starting from proper nouns in text documents containing information search, machine translation, answering questions, and automatic summarization. This study aims to apply NER in Indonesian language news classification. This study uses Design-Based Research whose process includes (1) pre-implementation, (2) design, (3) implementation and revision, and finally, (4) reflection and evaluation. This application was developed on the platform python, streamlit, BeautifulSoup, gnews, and spacy library. The results of application accuracy testing have an F1-score value of 89.69% for all entities consisting of place, figure, day, date, and organization.

Telugu Dependency Treebank

We discuss Telugu Language and Treebanks briefly in this work. Initially, we'll go over the Telugu language briefly. The paninian grammatical model utilized for Telugu dependency representation is then described. Following that, we explain Telugu treebanks and the various formats used to express these treebanks. We also discuss the Telugu language and its representation in the Telugu Dependency Treebank, and we give information on the Telugu language and the Telugu Dependency Treebank. Natural languages are often morphologically rich, and they create sentences in a variety of ways. Researchers have been investigating approaches to annotate text with linguistic knowledge since the advent of machine translation in the 1960s. Previous studies on Indian languages were done at the chunk level. The present shallow parser morphologically parses the input text to the chunk label. Researchers are considering working at the phrase level in the future. They broke the phrases down into smaller parts. The relationship between chunk heads is essential to proceed to sentence-level parsing. This results in reliance parsing.

Automatic Generation of Literary Sentences in French

In this paper, we describe a model for the automatic generation of literary sentences in French. Although there has been much recent effort directed towards automatic text generation in general, the generation of creative, literary sentences that is not restricted to a specific genre, which we approached in this work, is a difficult task that is not commonly treated in the scientific literature. In particular, our present model has not been previously applied to the generation of sentences in the French language. Our model was based on algorithms that we previously used to generate sentences in Spanish and Portuguese and on a new corpus, which we constructed and present here, consisting of literary texts in French, called MegaLitefr. Our automatic text generation algorithm combines language models, shallow parsing, the canned text method, and deep learning artificial neural networks. We also present a manual evaluation protocol that we propose and implemented to assess the quality of the artificial sentences generated by our algorithm, by testing if they fulfil four simple criteria. We obtained encouraging results from the evaluators for most of the desired features of our artificially generated sentences.

Multilabel Emotion Tagging for Domain-Specific Texts

In this article, we propose a novel approach to bootstrap a general seed emotion lexicon with words found in a domain-specific corpus. The approach divulges the contextual similarity between two words in the corpus via lexical-, dictionary-, and topic-based features, thus revealing the emotion labels of domain-specific words. As unfolding the recursive structure of language is an irreducible component of how humans understand a sentence, in this study, a propagation mechanism is designed that takes advantage of a shallow parser to derive the emotions associated with the words and their parent phrases. This mechanism pushes beyond the limits of most word co-occurrence approaches and facilitates the multilabel emotion tagging of a sentence in a manner reflecting human cognition. Evaluations on two benchmark corpora support the validity of the propagation mechanism. Further evaluation of a financial corpus indicates that our system outperforms the traditional bag-of-words approach. Our approach provides better modeling of compositional emotions by considering the emotion-bearing words, shifters, intensifiers, and overall sentence structure.

ASER: Towards large-scale commonsense knowledge acquisition via higher-order selectional preference over eventualities

Commonsense knowledge acquisition and reasoning have long been a core artificial intelligence problem. However, in the past, there has been a lack of scalable methods to collect commonsense knowledge. In this paper, we propose to develop principles for collecting commonsense knowledge based on selectional preference, which is a common phenomenon in human languages that has been shown to be related to semantics. We generalize the definition of selectional preference from one-hop linguistic syntactic relations to higher-order relations over linguistic graphs. Unlike previous commonsense knowledge definitions (e.g., ConceptNet), the selectional preference (SP) knowledge only relies on statistical distributions over linguistic graphs, which can be efficiently and accurately acquired from the unlabeled corpora with modern tools, rather than human-defined relations. As a result, acquiring SP knowledge is a much more scalable way of acquiring commonsense knowledge. Following this principle, we develop a large-scale eventuality (a linguistic term covering activity, state, and event)-based knowledge graph ASER, where each eventuality is represented as a dependency graph, and the relation between them is a discourse relation defined in shallow discourse parsing. The higher-order selectional preference over collected linguistic graphs reflects various kinds of commonsense knowledge. For example, dogs are more likely to bark than cats as the eventuality “dog barks” appears 14,998 times in ASER while “cat barks” only appears 6 times. “Be hungry” is more likely to be the reason rather than result of “eat food” as the edge 〈“be hungry,” Cause, “eat food”〉 appears in ASER while 〈“eat food,” Cause, “be hungry”〉 does not. Moreover, motivated by the observation that humans understand events by abstracting the observed events to a higher level and can thus transfer their knowledge to new events, we propose a conceptualization module on top of the collected knowledge to significantly boost the coverage of ASER. In total, ASER contains 648 million edges between 438 million eventualities. After conceptualization with Probase, a selectional preference based concept-instance relational knowledge base, our concept graph contains 15 million conceptualized eventualities and 224 million edges between them. Detailed analysis is provided to demonstrate its quality. All the collected data, APIs, and tools that can help convert collected SP knowledge into the format of ConceptNet are available at https://github.com/HKUST-KnowComp/ASER.

METHOD OF DOMAIN ONTOLOGY AUTOMATED REPLENISHMENT FOR THE SUPPORT OF NEW TECHNICAL SOLUTIONS SYNTHESIS. PART I

To solve the problem of information support for the synthesis of new technical solutions, a method of extracting structured data from an array of Russian-language patents is presented. The key features of the invention, such as the structural elements of the technical object and the relationships between them, are considered as information support. The data source addresses the main claim of the invention in the device patent. The unit of extraction is the semantic structure Subject-Action-Object (SAO), which semantically describes the constructive elements. The extraction method is based on shallow parsing and claim segmentation, taking into account the specifics of writing patent texts. Often the excessive length of the claim sentence and the specificity of the patent language make it difficult to efficiently use off-the-shelf tools for data extracting. All processing steps include: segmentation of the claim sentences; extraction of primary SAO structures; construction of the graph of the construct elements f the invention; integration of the data into the domain ontology. This article deals with the first two stages. Segmentation is carried out according to a number of heuristic rules, and several natural language processing tools are used to reduce analysis errors. The primary SAO elements are extracted considering the valences of the predefined semantic group of verbs, as well as information about the type of processed segment. The result of the work is the organization of the domain ontology, which can be used to find alternative designs for nodes in a technical object. In the second part of the article, an algorithm for constructing a graph of structural elements of a separate technical object, an assessment of the effectiveness of the system, as well as ontology organization and the result are considered.

Convolutional Neural Networks Naively Trained on Radiation Oncology-Specific DATA Outperforms Classical Natural Language Processing Approaches for Automated Identification of Common Toxicity Terms

Electronic medical records (EMR) hold potential for transformative improvements in the quality and efficiency of healthcare delivery. However, the unstructured nature of EMR data often necessitates manual review and is a significant barrier to leveraging it for downstream analysis. Methods to address this challenge include natural language processing (NLP) algorithms such as tokenization, shallow parsing, and boundary/negation detection. Recent radiation oncology (RO) specific attempts to use NLP to structure EMR data have involved identifying common toxicity terms in on-treatment visit (OTV) notes. Classical NLP works well for positive toxicity term identification (i.e., toxicity present) but is suboptimal for negated symptoms (i.e., toxicity absent). Convoluted neural networks (CNN) aid context detection, however, no publicly available RO-specific CNNs which identify toxicity terms exist. We hypothesized that RO-specific CNNs, naively trained on OTV notes, could improve tabulation of positive and negative toxicity information to an accuracy level suitable for automation.OTV notes (n = 3789) for prostate cancer (PCa) patients treated at our institution between 2019-2021 were identified and analyzed for inclusion/exclusion/omission of CTCAE toxicity terms. The top 5 terms identified (fatigue, nausea, diarrhea, dysuria, hematuria) were used for further analysis. Each note was manually classified by explicit positive identification, negation, or omission of each toxicity term, and used to train an in-house, toxicity-term-specific CNNs. Algorithms were structured as 3-group multiclass classification problem to distinguish between positive or negative symptom identification and omission of the term. Gold standard accuracy measurements were determined using manual review scores of OTV notes for the presence, absence or omission of toxicity terms in each OTV note. Overall, out of sample accuracy and F1 score were determined using a test/train split of OTV notes.The 3-class accuracy of CNNs for the top 5 CTCAE terms present/absent/negated in PCa OTV notes were: fatigue (accuracy = 0.93, F1 = 0.95), diarrhea (accuracy = 0.95, F1 = 0.94), nausea (accuracy = 0.98, F1 = 1.0), dysuria (accuracy = 0.97, F1 = 0.97), hematuria (accuracy = 0.99, F1 = 0.96).Training naïve CNNs with RO-specific training data from OTV notes increased the accuracy of CTCAE toxicity coding. This approach addresses challenges previously encountered using classical NLP from RO EMR data. Therefore, use of CNNs in NLP may reduce barriers to implementation of automated methods to improve data extraction for retrospective and prospective analyses.

Opinion Target Extraction Using a Shallow Semantic Parsing Framework

In this paper, we present a simplified shallow semantic parsing approach to extracting opinion targets. This is done by formulating opinion target extraction (OTE) as a shallow semantic parsing problem with the opinion expression as the predicate and the corresponding targets as its arguments. In principle, our parsing approach to OTE differs from the state-of-the-art sequence labeling one in two aspects. First, we model OTE from parse tree level, where abundant structured syntactic information is available for use, instead of word sequence level, where only lexical information is available. Second, we focus on determining whether a constituent, rather than a word, is an opinion target or not, via a simplified shallow semantic parsing framework. Evaluation on two datasets shows that structured syntactic information plays a critical role in capturing the domination relationship between an opinion expression and its targets. It also shows that our parsing approach much outperforms the state-of-the-art sequence labeling one.

Individual Chunking Ability Predicts Efficient or Shallow L2 Processing: Eye-Tracking Evidence From Multiword Units in Relative Clauses.

Behavioral studies on language processing rely on the eye-mind assumption, which states that the time spent looking at text is an index of the time spent processing it. In most cases, relatively shorter reading times are interpreted as evidence of greater processing efficiency. However, previous evidence from L2 research indicates that non-native participants who present fast reading times are not always more efficient readers, but rather shallow parsers. Because earlier studies did not identify a reliable predictor of variability in L2 processing, such uncertainty around the interpretation of reading times introduces a potential confound that undermines the credibility and the conclusions of online measures of processing. The present study proposes that a recently developed modulator of online processing efficiency, namely, chunking ability, may account for the observed variability in L2 online reading performance. L1 English – L2 Spanish learners’ eye movements were analyzed during natural reading. Chunking ability was predictive of overall reading speed. Target relative clauses contained L2 Verb-Noun multiword units, which were manipulated with regards to their L1-L2 congruency. The results indicated that processing of the L1-L2 incongruent units was modulated by an interaction of L2 chunking ability and level of knowledge of multiword units. Critically, the data revealed an inverse U-shaped pattern, with faster reading times in both learners with the highest and the lowest chunking ability scores, suggesting fast integration in the former, and lack of integration in the latter. Additionally, the presence of significant differences between conditions was correlated with individual chunking ability. The findings point at chunking ability as a significant modulator of general L2 processing efficiency, and of cross-language differences in particular, and add clarity to the interpretation of variability in the online reading performance of non-native speakers.

Learning Context-Aware Convolutional Filters for Implicit Discourse Relation Classification

Implicit discourse relation classification (IDRC) is considered the most difficult component of shallow discourse parsing as the relation prediction in the absence of necessary clues requires a deep understanding of the context information of the sentences. Convolutional neural networks (CNNs) have emerged as an important encoding block for sentences in natural language processing (NLP). CNNs use a specific set of filters for the inputs which may lead to the partial coverage of contextual clues. Furthermore, conventional CNNs may not allow the initial communication between the sentences which is a crucial step for IDRC. We present an adaptive convolution approach for IDRC that utilizes context aware filters for the convolution operation. The goal is to abstract the context of sentences in the filters and let them interact with sentence representations, i.e. learning the representations through learned filters. Our model acts as a cross questioning agent by generating filters from one argument and convolving them with the other for the IDRC task. This process is analogous to the attention mechanism because both methods aim at abstracting contextual information. Different from the attention mechanism, our approach directly encodes the contextual representations in the form of filters and allows the initial communication between arguments during encoding. Furthermore, the adaptive convolution can also work alongside the attention mechanism to enhance the representational ability of the adaptive CNN encoder. Experiments on PDTB 2.0 and CDTB datasets show that our approach outperforms all the baselines by a fair margin and achieves excellent results.

New approach to the chunk recoginition in Polish

This paper discusses the problem of shallow parsing of Polish, most specifically — chunking. We describe the linguistic work on annotation guidelines development, manual corpus annotation, and preparing the neural models used for chunking - the first one for the Polish language, and conducted the evaluation of these models. Finally, we describe conclusions and plans for future work.

A novel approach for automatic Bengali question answering system using semantic similarity analysis

Finding the semantically accurate answer is one of the key challenges in advanced searching. In contrast to keyword-based searching, the meaning of a question or query is important here and answers are ranked according to relevance. It is very natural that there is almost no common word between the question sentence and the answer sentence. In this paper, an approach is described to find out the semantically relevant answers in the Bengali dataset. In the first part of the algorithm, a set of statistical parameters like frequency, index, part-of-speech (POS) is matched between a question and the probable answers. In the second phase, entropy and similarity are calculated in different modules. Finally, a sense score is generated to rank the answers. The algorithm is tested on a repository containing a total of 275,000 sentences. This Bengali repository is a product of Technology Development for Indian Languages (TDIL) project sponsored by Govt. of India and provided by the Language Research Unit of Indian Statistical Institute, Kolkata. The shallow parser, developed by the LTRC group of IIIT Hyderabad is used for POS tagging. The actual answer is ranked as 1st in 82.3% cases. The actual answer is ranked within 1st to 5th in 90.0% cases. The accuracy of the system is coming as 97.32% and precision of the system is coming as 98.14% using confusion matrix. The challenges and pitfalls of the work are reported at last in this paper.

Representation learning in discourse parsing: A survey

Neural network based deep learning methods aim to learn representations of data and have produced state-of-the-art results in many natural language processing (NLP) tasks. Discourse parsing is an important research topic in discourse analysis, aiming to infer the discourse structure and model the coherence of a given text. This survey covers text-level discourse parsing, shallow discourse parsing and coherence assessment. We first introduce the basic concepts and traditional approaches, and then focus on recent advances in discourse structure oriented representation learning. We also introduce a trend of discourse structure aware representation learning that is to exploit discourse structures or discourse objectives for learning representations of sentences and documents for specific applications or for general purpose. Finally, we present a brief summary of the progress and discuss several future directions.

A Comparative Study of Deep and Shallow Parsing Approaches to Automated Grammaticality Evaluation

The concept of automated grammar evaluation of natural language texts is one that has attracted significant interests in the natural language processing community. It is the examination of natural language text for grammatical accuracy using computer software. The current work is a comparative study of different deep and shallow parsing techniques that have been applied to lexical analysis and grammaticality evaluation of natural language texts. The comparative analysis was based on data gathered from numerous related works. Shallow parsing using induced grammars was first examined along with its two main sub-categories, the probabilistic statistical parsers and the connectionist approach using neural networks. Deep parsing using handcrafted grammar was subsequently examined along with several of it‟s subcategories including Transformational Grammars, Feature Based Grammars, Lexical Functional Grammar (LFG), Definite Clause Grammar (DCG), Property Grammar (PG), Categorial Grammar (CG), Generalized Phrase Structure Grammar (GPSG), and Head-driven Phrase Structure Grammar (HPSG). Based on facts gathered from literature on the different aforementioned formalisms, a comparative analysis of the deep and shallow parsing techniques was performed. The comparative analysis showed among other things that while the shallow parsing approach was usually domain dependent, influenced by sentence length and lexical frequency and employed machine learning to induce grammar rules, the deep parsing approach were not domain dependent, not influenced by sentence length nor lexical frequency, and they made use of well spelt out set of precise linguistic rules. The deep parsing techniques proved to be a more labour intensive approach while the induced grammar rules were usually faster and reliability increased with size, accuracy and coverage of training data. The shallow parsing approach has gained immense popularity owing to availability of large corpora for different languages, and has therefore become the most accepted and adopted approach in recent times.
 Keywords: Grammaticality, Natural language processing, Deep parsing, Shallow parsing, Handcrafted grammar, Precision grammar, Induced grammar, Automated scoring, Computational linguistics, Comparative study.

Rectifying Incorrectly Part of Speech-Tagged Polysemy Words in Kannada Language for Machine Translation

Polysemy words refer to the same word, but have different context, when used in a single sentence, resulting in variant Part of Speech (POS). Occurrence of these Polysemy words, more than once, in a Kannada sentences, leads to ambiguity and is often cumbersome to translate it to English, due to incorrect interpretation of the sentence. The algorithm proposed is, concentrated on a few examples, which can identify the wrong POS-tagged word in a sentence. The POS tagged sentence is obtained as an input through Shallow parser, and then, by owing to the structure of Kannada language, the algorithm identifies the incorrectly tagged word.