Part-of-speech Tagging Research Articles

Enhanced sequence labeling based on latent variable conditional random fields

Natural language processing is a useful processing technique of language data, such as text and speech. Sequence labeling represents the upstream task of many natural language processing tasks, such as machine translation, text classification, and sentiment classification. In this paper, the focus is on the sequence labeling task, in which semantic labels are assigned to each unit of a given input sequence. Two frameworks of latent variable conditional random fields (CRF) models (called LVCRF-I and LVCRF-II) are proposed, which use the encoding schema as a latent variable to capture the latent structure of the hidden variables and the observed data. Among the two designed models, the LVCRF-I model focuses on the sentence level, while the LVCRF-II works in the word level, to choose the best encoding schema for a given input sequence automatically without handcraft features. In the experiments, the two proposed models are verified by four sequence prediction tasks, including named entity recognition (NER), chunking, reference parsing and POS tagging. The proposed frameworks achieve better performance without using other handcraft features than the conventional CRF model. Moreover, these designed frameworks can be viewed as a substitution of the conventional CRF models. In the commonly used LSTM-CRF models, the CRF layer can be replaced with our proposed framework as they use the same training and inference procedure. The experimental results show that the proposed models exhibit latent variable and provide competitive and robust performance on all three sequence prediction tasks.

Natural language processing-enhanced extraction of SBVR business vocabularies and business rules from UML use case diagrams

Discovery, specification and proper representation of various aspects of business knowledge plays crucial part in model-driven information systems engineering, especially when it comes to the early stages of systems development. Being among the most applicable and advanced features of model-driven development, model transformation could help improving one of the most time- and resource-consuming efforts in this process, namely, discovery and specification of business vocabularies and business rules within the problem domain. One of our latest developments in this area was the solution for the automatic extraction of SBVR business vocabularies and business rules from UML use case diagrams, which was arguably one of the most comprehensive developments of this kind currently available in public. In this paper, we present an enhancement to our previous development by introducing a novel natural language processing component to it. This enhancement provides more advanced extraction capabilities (such as recognition of entities, entire noun and verb phrases, multinary associations) and better quality of the extraction results compared to our previous solution. The main contributions presented in this paper are pre- and post-processing algorithms, and two extraction algorithms using custom-trained POS tagger. Based on the related work findings, it is safe to state that the presented solution is novel and original in its approach of combining together M2M transformation of UML and SBVR models with natural language processing techniques in the field of model-driven information systems engineering.

A Part-Of-Speech Tagging Approach for Chinese-Hmong Mixed Text

Part-of-speech (POS) tagging is a basic problem that needs to be solved in the informationization of Chinese-Hmong mixed text including square Hmong characters, so far no one has studied it. This paper proposes a POS tagging approach for Chinese-Hmong mixed text by utilizing improved Hidden Markov Model (HMM) to expand contextual information. The results of comparative experiments based on cross-validation reveal the proposed approach has perfect performance, and is able to obtain tagging results with good consistency and high coverage.

Aspect based Sentiment Analysis of Employee’s Review Experience

Background: Employees of technology companies evaluate their experience through online reviews. Online reviews of companies from employees or former employees help job seeker to find out the weaknesses and strengths of the companies. The reviews can be used as an evaluation tool for each technology company to understand their employee’s perceptions. However, most information on online reviews is not well responded since some of the detailed information of the company is missing. Objective: This study aims to generate an Aspect-based Sentiment Analysis using user review data. The review data were then extracted and classified into five aspects: work balance, culture value, career opportunities, company benefit, and management. The output of this study is the aspect score from each company.Methods: This study suggests a method to analyze online reviews from employees in detail, so it can prevent the missing of specific information. The analysis was sequentially carried out in five stages. First, user review data were crawled from Glassdoor and stored in a database. Second, the raw data were processed in the data pre-processing stage to delete the incomplete data. Third, the words other than noun keyword were eliminated using Standford POS Tagger. Fourth, the noun keywords were then classified into each aspect. Finally, the aspect score was calculated based on the aspect-based sentiment analysis.Results: Result showed that the proposed method managed to turn raw review data into five aspects based on user perception.Conclusion: The study provides information for two parties, job seeker and the company. The analysis of the review could help the job seeker to decide which company that suits his need and ability. For the companies, it can be a great assistance because they will be more aware of their strengths and weaknesses. This study could possibly also provide ratings to the companies based on the aspects that have been determined.

A simple Galois Power-of-Two real time embedding scheme for performing Arabic morphology deep learning tasks

This paper describes how a simple novel Galois Power-of-Two (GPOW2) real-time embedding scheme is used to improve the performance and accuracy of downstream NLP tasks. GPOW2 computes embeddings live on the fly (real time) in the context of target NLP tasks without the need for tabulated pre-embeddings. One excellent feature of the method is the ability to capture multilevel embeddings in the same pass. It simultaneously computes character, word and sentence embeddings on the fly. GPOW2 has been derived in the context of attempts to improve the performance of the SWAM Arabic morphological engine, which is a multipurpose tool that supports segmentation, classification, POS tagging, spell checking, word embeddings, sematic search, among other tasks. SWAM is a pattern-oriented algorithm that relies on morphological patterns and POS tagging to perform NLP tasks. The paper demonstrates how GPOW2 led to improvements in the accuracy of POS tagging and pattern matching, and accordingly the performance of the whole engine. The accuracy for pattern prediction is 99.47% and is 98.80% for POS tagging.

Multi-source social media data sentiment analysis using bidirectional recurrent convolutional neural networks

Subjectivity detection in the text is essential for sentiment analysis, which requires many techniques to perceive unanticipated means of communication. Few accomplishments adapted to capture the syntactic, semantic, and contextual sentimental information via distributed word representations (DWRs)11Distributed work representations .. This paper, concatenating the DWRs through a weighted mechanism on Recurrent Neural Network (RNN) variants joint with Convolutional Neural network (CNN) distinctively involving weighted attentive pooling (WAP)22Weighted Attentive Pooling .. Whereas, CNNs with traditional pooling operations comprise many layers merely able to capture enough features. Our considerations empower the sentiment analysis over DWRs contains Word2vec, FastText, and GloVe to produce dense efficient concatenated representation (DECR)33Dense Efficient Concatenated Representation . to hold long term dependencies on a single RNN layer acquired by Parts of Speech Tagging (POS) explicitly with verbs, adverbs, and noun only. Then use these representations gained in a way, inputted to CNN contain single convolution layer engaging WAP on multi-source social media data to handle the issues of syntactic and semantic regularities as well as out of vocabulary (OOV) words. Experimentations demonstrate that DWRs together with proposed concatenation qualified in resolving the mentioned issues by moderate hyper-parameter configurations. Our architecture devoid of stacking multiple layers achieved modest accuracy of 89.67% by DECR-Bi-GRU-CNN (WAP) on IMDB as compared to random initialization 81.11% on SST.

Interactive Neural Network: Leveraging Part-of-Speech Window for Aspect Term Extraction (Student Abstract)

Aspect term extraction is a fundamental task for aspect-level sentiment analysis. Previous methods tend to extract noun aspect terms due to the large quantities of them, and perform badly on extracting aspect terms containing words with other POS tags, according to experimental results. In addition, few works focus on the POS tags of adjacent words which are critical to aspect term extraction. We propose a novel model which combines POS and word features in an interactive way, and makes full use of the POS tags of adjacent words by POS window. We conduct experiments on two datasets, and prove the effectiveness of our model.

Weakly Supervised POS Taggers Perform Poorly on <em>Truly</em> Low-Resource Languages

Part-of-speech (POS) taggers for low-resource languages which are exclusively based on various forms of weak supervision – e.g., cross-lingual transfer, type-level supervision, or a combination thereof – have been reported to perform almost as well as supervised ones. However, weakly supervised POS taggers are commonly only evaluated on languages that are very different from truly low-resource languages, and the taggers use sources of information, like high-coverage and almost error-free dictionaries, which are likely not available for resource-poor languages. We train and evaluate state-of-the-art weakly supervised POS taggers for a typologically diverse set of 15 truly low-resource languages. On these languages, given a realistic amount of resources, even our best model gets only less than half of the words right. Our results highlight the need for new and different approaches to POS tagging for truly low-resource languages.

Keyword extraction and structuralization of medical reports.

In recent years, patients usually accept more accurate and detailed examinations because of the rapid advances in medical technology. Many of the examination reports are not represented in numerical data, but text documents written by the medical examiners based on the observations from the instruments and biochemical tests. If the above-mentioned unstructured data can be organized as a report in a structured form, it will help doctors to understand a patient's status of the various examinations more efficiently. Besides, further association analysis on the structuralized data can be performed to identify potential factors that affect a disease. In this paper, from the pathology examination reports of renal diseases, we applied the POS tagging results of natural language analysis to automatically extract the keyword phrases. Then a medical dictionary for various examination items in an examination report is established, which is used as the basic information for retrieving the terms to construct a structured form of the report. Moreover, a topical probability modeling method is applied to automatically discover the candidate keyword phrases of the examination items from the reports. Finally, a system is implemented to generate the structured form for the various examination items in a report according to the constructed medical dictionary. The results of the experiments showed that the methods proposed in this paper can effectively construct a structural form of examination reports. Furthermore, the keywords of the popular examination items can be extracted correctly. The above techniques will help automatic processing and analysis of medical text reports.

Improving accuracy of Part-of-Speech (POS) tagging using hidden markov model and morphological analysis for Myanmar Language

In Natural Language Processing (NLP), Word segmentation and Part-of-Speech (POS) tagging are fundamental tasks. The POS information is also necessary in NLP’s preprocessing work applications such as machine translation (MT), information retrieval (IR), etc. Currently, there are many research efforts in word segmentation and POS tagging developed separately with different methods to get high performance and accuracy. For Myanmar Language, there are also separate word segmentors and POS taggers based on statistical approaches such as Neural Network (NN) and Hidden Markov Models (HMMs). But, as the Myanmar language's complex morphological structure, the OOV problem still exists. To keep away from error and improve segmentation by utilizing POS data, segmentation and labeling should be possible at the same time.The main goal of developing POS tagger for any Language is to improve accuracy of tagging and remove ambiguity in sentences due to language structure. This paper focuses on developing word segmentation and Part-of- Speech (POS) Tagger for Myanmar Language. This paper presented the comparison of separate word segmentation and POS tagging with joint word segmentation and POS tagging.

Performance Evaluation of English Part-of-Speech Tagging Based on Typical Parameter Smoothing Algorithm

Part-of-speech (POS) tagging for English is the basis for implementing English automatic correction. Although researchers have done a lot of useful studies on English POS tagging, most of them are aimed at users with English as the first language, while studies for users with English as the second language are few. For this purpose, manual tagging is performed based on the typical parameter smoothing algorithm. On this basis, a performance evaluation method for English part-of-speech tagging is proposed, which integrates the features of term clustering, non-tagged corpus statistics, word pronunciation, etc. The experimental results show that the algorithm can improve the performance of POS tagging effectively, with the tagging accuracy improved from 94.49% to 97.07%

Establishing the Syntactic Rules of the Kankanaey Dialect using RNN

The diverse culture and ethnic groups in the Philippines creates a beautiful mixture of ideas, traditions, and practices but also makes it hard for researchers to keep track of them all. One integral part of any culture is language, with one of the most spoken languages in the Cordillera Administrative Region (CAR) being Kankanaey. Unfortunately, it has very little resources and documentation for it. This paper presents a corpus created for Kankanaey that contains 3412 words and was trained with a dataset containing 400 Kankanaey sentences in order to establish its syntactic rules. Data for the collected texts for Kankanaey were taken from public sources online and were organized into various categories based on the type of content. Training and testing was done to establish the syntactic rules using the Keras API. The rules were derived by having each word in the training sentences tagged with the corresponding POS tag. After tagging, the number of POS tags were then expanded to all possible combinations of the POS which resulted in the documenting of 1,722 syntactic rules for Kankanaey with the model having an accuracy of 64% when it was tested to identify the syntactic rules in 50 test sentences.

Human Behavior Prediction based on Opinions using Machine Learning Techniques

Prediction is the way of identifying the behavior of a person towards online shopping by analyzing the reviews publicly available on the web. In the present study, machine learning approaches are used to extract reviews from the web and segregate and classify them in to five categories, namely, strongly positive, positive, neutral, negative, and strongly negative, for the prediction of human behavior. Several pre-processing methods (including stop-word removal) are applied and web crawler is used to gather the data. This is followed by the application of Stanford POS tagger for tagging the reviews, which is done after stemming by using the porter stemmer algorithm. Analysis of a person’s behavior is performed and experimental results are compared with machine learning approaches.

Improving Code-mixed POS Tagging Using Code-mixed Embeddings

Social media data has become invaluable component of business analytics. A multitude of nuances of social media text make the job of conventional text analytical tools difficult. Code-mixing of text is a phenomenon prevalent among social media users, wherein words used are borrowed from multiple languages, though written in the commonly understood roman script. All the existing supervised learning methods for tasks such as Parts Of Speech (POS) tagging for code-mixed social media (CMSM) text typically depend on a large amount of training data. Preparation of such large training data is resource-intensive, requiring expertise in multiple languages. Though the preparation of small dataset is possible, the out of vocabulary (OOV) words pose major difficulty, while learning models from CMSM text as the number of different ways of writing non-native words in roman script is huge. POS tagging for code-mixed text is non-trivial, as tagging should deal with syntactic rules of multiple languages. The important research question addressed by this article is whether abundantly available unlabeled data can help in resolving the difficulties posed by code-mixed text for POS tagging. We develop an approach for scraping and building word embeddings for code-mixed text illustrating it for Bengali-English, Hindi-English, and Telugu-English code-mixing scenarios. We used a hierarchical deep recurrent neural network with linear-chain CRF layer on top of it to improve the performance of POS tagging in CMSM text by capturing contextual word features and character-sequence–based information. We prepared a labeled resource for POS tagging of CMSM text by correcting 19% of labels from an existing resource. A detailed analysis of the performance of our approach with varying levels of code-mixing is provided. The results indicate that the F1-score of our approach with custom embeddings is better than the CRF-based baseline by 5.81%, 5.69%, and 6.3% in Bengali, Hindi , and Telugu languages, respectively.

A Hybrid Method of Linguistic and Statistical Features for Arabic Sentiment Analysis

Sentiment analysis refers to the task of identifying polarity of positive and negative for particular text that yield an opinion. Arabic language has been expanded dramatically in the last decade especially with the emergence of social websites (e.g. Twitter, Facebook, etc.). Several studies addressed sentiment analysis for Arabic language using various techniques. The most efficient techniques according to the literature were the machine learning due to their capabilities to build a training model. Yet, there is still issues facing the Arabic sentiment analysis using machine learning techniques. Such issues are related to employing robust features that have the ability to discriminate the polarity of sentiments. This paper proposes a hybrid method of linguistic and statistical features along with classification methods for Arabic sentiment analysis. Linguistic features contains stemming and POS tagging, while statistical contains the TF-IDF. A benchmark dataset of Arabic tweets have been used in the experiments. In addition, three classifiers have been utilized including SVM, KNN and ME. Results showed that SVM has outperformed the other classifiers by obtaining an f-score of 72.15%. This indicates the usefulness of using SVM with the proposed hybrid features.

Fast paraphrase extraction in Ancient Greek literature

Abstract In this paper, A shorter version of the paper appeared in German in the final report of the Digital Plato project which was funded by the Volkswagen Foundation from 2016 to 2019. [35], [28]. we present a method for paraphrase extraction in Ancient Greek that can be applied to huge text corpora in interactive humanities applications. Since lexical databases and POS tagging are either unavailable or do not achieve sufficient accuracy for ancient languages, our approach is based on pure word embeddings and the word mover’s distance (WMD) [20]. We show how to adapt the WMD approach to paraphrase searching such that the expensive WMD computation has to be computed for a small fraction of the text segments contained in the corpus, only. Formally, the time complexity will be reduced from O ( N · K 3 · log K ) \mathcal{O}(N\cdot {K^{3}}\cdot \log K) to O ( N + K 3 · log K ) \mathcal{O}(N+{K^{3}}\cdot \log K) , compared to the brute-force approach which computes the WMD between each text segment of the corpus and the search query. N is the length of the corpus and K the size of its vocabulary. The method, which searches not only for paraphrases of the same length as the search query but also for paraphrases of varying lengths, was evaluated on the Thesaurus Linguae Graecae® (TLG®) [25]. The TLG consists of about 75 · 10 6 75\cdot {10^{6}} Greek words. We searched the whole TLG for paraphrases for given passages of Plato. The experimental results show that our method and the brute-force approach, with only very few exceptions, propose the same text passages in the TLG as possible paraphrases. The computation times of our method are in a range that allows its application in interactive systems and let the humanities scholars work productively and smoothly.

POS tagger model for Kannada text with CRF++ and deep learning approaches

Computational Linguistics is one of the interesting topics in the research field of Computer Science. This paper presents training for Part of Speech (POS) tagging on Kannada words using two techniques. First approach is supervised machine learning technique CRF++0.50 (Conditional Random Field). The second approach is a combination of word embedding and deep learning techniques. The total dataset used for this implementation is 1200 tagged Kannada sentences downloaded from Technology Development for Indian Languages (TDIL). We divided the dataset into 1100 sentences (13,600 words) as training data and 100 sentences (1053 words) as test data. The BIS (Bureau of Indian Standards) tagset is used in this work in which 27 major POS tags have been considered. An accuracy obtained through CRF++0.50 tool is 76% and that with deep learning technique is 71%. The precision, recall and f-score of each tag using both the techniques are considerable.

Detection Method of Data Integrity in Network Storage Based on Symmetrical Difference

In order to enhance the recall and the precision performance of data integrity detection, a method to detect the network storage data integrity based on symmetric difference was proposed. Through the complete automatic image annotation system, the crawler technology was used to capture the image and related text information. According to the automatic word segmentation, pos tagging and Chinese word segmentation, the feature analysis of text data was achieved. Based on the symmetrical difference algorithm and the background subtraction, the feature extraction of image data was realized. On the basis of data collection and feature extraction, the sentry data segment was introduced, and then the sentry data segment was randomly selected to detect the data integrity. Combined with the accountability scheme of data security of the trusted third party, the trusted third party was taken as the core. The online state judgment was made for each user operation. Meanwhile, credentials that cannot be denied by both parties were generated, and thus to prevent the verifier from providing false validation results. Experimental results prove that the proposed method has high precision rate, high recall rate, and strong reliability.

Corpus-based Study of Identifying Verb Patterns Used in Pakistani Newspaper Headlines

Newspaper headlines are an important subgenre of media genre and enjoy much significance in news discourse. Headlines are ascribed different functions as they are the opening section to their respective text. This corpus-driven study strives to identify those verb patterns which have been used in Pakistani newspaper headlines. To identify different verb patterns used in newspaper headlines, the researcher compiled a corpus of 3135 newspaper headlines consisting of 28646 words drawn from three on-line Pakistani English newspapers which include The Dawn, The Nation and The News. The researcher tagged this corpus by using the software TagAnt and analyzed this corpus with the help of corpus tool AntConc to identify the verb patterns used in these Pakistani English newspaper headlines. To this end, the researcher analyzed the compiled corpus in accordance with the POS Tags given by Tree Tagger Tag Set. This study has found different verb patterns which have been used in newspaper headlines.

Viability of Neural Networks for Core Technologies for Resource-Scarce Languages

In this paper, the viability of neural network implementations of core technologies (the focus of this paper is on text technologies) for 10 resource-scarce South African languages is evaluated. Neural networks are increasingly being used in place of other machine learning methods for many natural language processing tasks with good results. However, in the South African context, where most languages are resource-scarce, very little research has been done on neural network implementations of core language technologies. In this paper, we address this gap by evaluating neural network implementations of four core technologies for ten South African languages. The technologies we address are part of speech tagging, named entity recognition, compound analysis and lemmatization. Neural architectures that performed well on similar tasks in other settings were implemented for each task and the performance was assessed in comparison with currently used machine learning implementations of each technology. The neural network models evaluated perform better than the baselines for compound analysis, are viable and comparable to the baseline on most languages for POS tagging and NER, and are viable, but not on par with the baseline, for Afrikaans lemmatization.