Raw Text Corpora Research Articles

Verb Classification Across Languages

Recent developments in language modeling have enabled large text encoders to derive a wealth of linguistic information from raw text corpora without supervision. Their success across natural language processing (NLP) tasks has called into question the role of man-made computational resources, such as verb lexicons, in supporting modern NLP. Still, probing analyses have concurrently exposed the limitations of the knowledge possessed by the large neural architectures, revealing them to be clever task solvers rather than self-taught linguists. Can human-designed lexical resources still help fill their knowledge gaps? Focusing on verb classification, we discuss approaches to generating verb classes multilingually and weigh the relative benefits of undertaking expensive lexicographic work and outsourcing the task to untrained native speakers. Then, we consider the evidence for the utility of augmenting pretrained language models with external verb knowledge and ponder the ways in which human expertise can continue to benefit multilingual NLP.

Distributed Representation of Words in Cause and Effect Spaces

This paper focuses on building up distributed representation of words in cause and effect spaces, a task-specific word embedding technique for causality. The causal embedding model is trained on a large set of cause-effect phrase pairs extracted from raw text corpus via a set of high-precision causal patterns. Three strategies are proposed to transfer the positive or negative labels from the level of phrase pairs to the level of word pairs, leading to three causal embedding models (Pairwise-Matching, Max-Matching, and AttentiveMatching) correspondingly. Experimental results have shown that Max-Matching and Attentive-Matching models significantly outperform several state-of-the-art competitors by a large margin on both English and Chinese corpora.

Learning for clinical named entity recognition without manual annotations

Abstract Background Named entity recognition (NER) systems are commonly built using supervised methods that use machine learning to learn from corpora manually annotated with named entities. However, manually annotating corpora is very expensive and laborious. Materials and methods In this paper, a novel method is presented for training clinical NER systems that does not require any manual annotations. It only requires a raw text corpus and a resource like UMLS that can give a list of named entities along with their semantic types. Using these two resources, annotations are automatically obtained to train machine learning methods. The method was evaluated on the NER shared-task datasets of i2b2 2010 and SemEval 2014. Results On the SemEval 2014 dataset for recognizing diseases and disorders, the method obtained F-measure of 0.693 for exact matching and of 0.773 allowing overlaps. This is comparable to many supervised systems in the past that had used manual annotations for training. On the i2b2 2010 dataset for recognizing problems, tests and treatments, the method obtained F-measures of 0.451, 0.338 and 0.204 respectively for exact matching and of 0.721, 0.587 and 0.475 respectively allowing overlaps. These results are better than an existing unsupervised method. Conclusions Experiments on standard datasets showed that the new method performed well. The method is general and could be applied to recognize entities of other types on other genres of text without needing manual annotations.

Word Sense Induction with Closed Frequent Termsets

The article is devoted to the problem of word sense induction. We propose a method for inducing senses from a raw text corpus. The proposed sense induction algorithm (called SenseSearcher, or SnS) is based on closed frequent sets, and as a result, it provides a multilevel sense representation. To a large extent, it is a knowledge‐poor approach, as it does not need any kind of structured knowledge base about senses and there is no deep language knowledge embedded. By discovering a hierarchy of senses, the algorithm enables identifying subsenses (fine‐grained senses). SnS discovers not only frequent (dominating) senses but also infrequent ones (dominated). The method was evaluated in two main areas: lexicography and information retrieval. With the use of the SnS algorithm, we provide a tool able to induce from a textual corpus a structure of senses, with a varying number of granularity levels. In the area of information retrieval, SnS can be used for clustering search result, according to the discovered senses. The experiments have shown that SnS performs better than the methods participating in the SemEval2013 WSI Task 11 competition, and most of the known search result clustering methods.

Unsupervised Learning and Linguistic Rule Based Algorithm for Uyghur Word Segmentation

Inter-word spaces based traditional word segmentation method not very appropriate for multi-word structured semantic words due to the fact that it will split the semantic words into several fragments that inconsistent with its original meaning. So, this will be a bottleneck problem in Uyghur text analysis and text understanding applications. This paper puts forward a new idea and related algorithms for segmentation of Uyghur multiword structured semantic words. In this algorithm, the word based Bi-gram and contextual information are derived from large scale raw text corpus automatically, and according to the association rules between Uyghur words, the liner combinations of mutual information, difference of t-test and dual adjacent entropy are taken as a new measurement( dmd ) to estimate the agglutinative strength between two adjacent Uyghur words. The experimental result on large-scale open tests shows that the proposed algorithm achieves 88.21% segmentation accuracy

Dynamic Programming Method Applied in Vietnamese Word Segmentation Based on Mutual Information among Syllables

Vietnamese word segmentation is an important step in Vietnamese natural language processing such as text categorization, text summary, and automated machine translation. The problem with Vietnamese word segmentation is complicated because Vietnamese words are not always separated by a space. One word can include one or more syllables depending on the context. This paper proposes a method for Vietnamese word segmentation based on the mutual information among the syllables combined with dynamic programming. With this method, we can achieve an accuracy rate of about 90% with a raw text corpus.

Unsupervised word sense disambiguation for Korean through the acyclic weighted digraph using corpus and dictionary

Word sense disambiguation (WSD) is meant to assign the most appropriate sense to a polysemous word according to its context. We present a method for automatic WSD using only two resources: a raw text corpus and a machine-readable dictionary (MRD). The system learns the similarity matrix between word pairs from the unlabeled corpus, and it uses the vector representations of sense definitions from MRD, which are derived based on the similarity matrix. In order to disambiguate all occurrences of polysemous words in a sentence, the system separately constructs the acyclic weighted digraph (AWD) for every occurrence of polysemous words in a sentence. The AWD is structured based on consideration of the senses of context words which occur with a target word in a sentence. After building the AWD per each polysemous word, we can search the optimal path of the AWD using the Viterbi algorithm. We assign the most appropriate sense to the target word in sentences with the sense on the optimal path in the AWD. By experiments, our system shows 76.4% accuracy for the semantically ambiguous Korean words.

Deep lexical acquisition of verb–particle constructions

This paper proposes a range of techniques for extracting English verb-particle constructions from raw text corpora, complete with valence information. We propose four basic methods, based on the output of a POS tagger, chunker, chunk grammar and dependency parser, respectively. We then present a combined classifier which we show to consolidate the strengths of the component methods.

DOMAIN KNOWLEDGE ACQUISITION AND PLAN RECOGNITION BY PROBABILISTIC REASONING

This paper introduces a statistical framework for extracting medical domain knowledge from heterogeneous corpora. The acquired information is incorporated into a natural language understanding agent and applied to DIKTIS, an existing web-based educational dialogue system for the chemotherapy of nosocomial and community acquired pneumonia, aiming at providing a more intelligent natural language interaction. Unlike the majority of existing dialogue understanding engines, the presented system automatically encodes semantic representation of a user's query using Bayesian networks. The structure of the networks is determined from annotated dialogue corpora using the Bayesian scoring method, thus eliminating the tedious and costly process of manually coding domain knowledge. The conditional probability distributions are estimated during a training phase using data obtained from the same set of dialogue acts. In order to cope with words absent from our restricted dialogue corpus, a separate offline module was incorporated, which estimates their semantic role from both medical and general raw text corpora, correlating them with known lexical-semantically similar words or predefined topics. Lexical similarity is identified on the basis of both contextual similarity and co-occurrence in conjunctive expressions. The evaluation of the platform was performed against the existing language natural understanding module of DIKTIS, the architecture of which is based on manually embedded domain knowledge.

A language model using variable length tokens for open-vocabulary Hangul text recognition

We propose a novel language model for Hangul text recognition. Without relying on prior linguistic knowledge in training, the proposed model learns variable length Hangul character sequences, which comprise the elementary tokens of Korean language, and their probabilities from statistics of a raw text corpus. Experiments in handwritten Hangul recognition shows that the proposed language model is effective in postprocessing of recognition results.