Collocation Extraction Research Articles

Violeta Seretan. Syntax-Based Collocation Extraction

This book tackles the question of Syntax-Based Collocation Extraction from a computational perspective. It first gives an overview of collocation studies over time and then details the creation of a collocation extraction tool developed by the author during her thesis at the University of Geneva. It is a relatively short book consisting of 127 of text pages divided into six chapters, including both the introduction and conclusion, and with six appendices, an extensive bibliography and an index covering a further 89 pages.

Extracting Collocations from Bengali Text Corpus

Automatic collocation extraction is very important in various applications in the field of natural language processing such as machine translation, word sense disambiguation, information retrieval, and language modelling in speech processing, lexicography and many more. The success of extracting collocations depends on the technique of preprocessing. A systematic pre-processing technique is described in this paper. Then the pre-processed data is used to extract collocation by using two methods: Point-wise Mutual Information and Fuzzy Bi-gram Index. The paper mainly focuses on bi-gram extraction from a Bengali news corpus. Collocations of higher length i.e., n-grams (n>2) are then obtained when the extracted collocations of lower lengths are treated as individual words.

Two-Word Collocation Extraction Using Monolingual Word Alignment Method

Statistical bilingual word alignment has been well studied in the field of machine translation. This article adapts the bilingual word alignment algorithm into a monolingual scenario to extract collocations from monolingual corpus, based on the fact that the words in a collocation tend to co-occur in similar contexts as in bilingual word alignment. First, the monolingual corpus is replicated to generate a parallel corpus, in which each sentence pair consists of two identical sentences. Next, the monolingual word alignment algorithm is employed to align potentially collocated words. Finally, the aligned word pairs are ranked according to the alignment scores and candidates with higher scores are extracted as collocations. We conducted experiments on Chinese and English corpora respectively. Compared to previous approaches that use association measures to extract collocations from co-occurrence word pairs within a given window, our method achieves higher precision and recall. According to human evaluation, our method achieves precisions of 62% on a Chinese corpus and 64% on an English corpus. In particular, we can extract collocations with longer spans, achieving a higher precision of 83% on the long-span (> 6 words) Chinese collocations.

Measuring Collocation Tendency of Words

In all natural languages, some words collocate with other words to create multi-worded blocks of meaning – the collocations. Since identification of collocations is vital for information retrieval, language learning, psycholinguistics, authorship determination and translation, collocation extraction is an important issue in natural language processing. In this paper we present a method which is designed to improve current statistical methods that generate ranked lists of collocation candidates. Due to meaning integrity, any word in a collocation must suggest or at least imply the subsequent words composing the collocation. As a result, we may state that the words in a random text differ in the tendency to facilitate the prediction of the next word. If a word helps the prediction then it tends to collocate, otherwise it does not. In this paper, an attempt has been made to extract collocations by measuring collocation tendency of words and word combinations. The method used is to filter out free word pairs (the words that do not facilitate the prediction of the next word or those in which meaning integrity has not been completed yet) in the lists of candidate pairs. Collocation tendency method is tested on a base data set extracted by some statistical collocation extraction techniques (frequency of occurrence, point-wise mutual information, the t-test, chi-square techniques) and is evaluated by precision and recall measures. We have found that collocation tendency method brings a remarkable improvement on frequency of occurrence and the t-test techniques.

An Automatic Collocation Extraction from Arabic Corpus

Problem statement: The identification of collocations is very important part in natural language processing applications that require some degree of semantic interpretation such as, machine translation, information retrieval and text summarization. Because of the complexities of Arabic, the collocations undergo some variations such as, morphological, graphical, syntactic variation that constitutes the difficulties of identifying the collocation. Approach: We used the hybrid method for extracting the collocations from Arabic corpus that is based on linguistic information and association measures. Results: This method extracted the bi-gram candidates of Arabic collocation from corpus and evaluated the association measures by using the n-best evaluation method. We reported the precision values for each association measure in each n-best list. Conclusion: The experimental results showed that the log-likelihood ratio is the best association measure that achieved highest precision.

Fuzzy Set Theoretic Approach To Collocation Extraction

Fuzzy approach deals with the linguistic properties of elements such as beauty, coldness, hotness etc. Collocations are linguistically motivated. Decision of word combination for being collocation is a linguistic term as merely co occurrence of word combinations does not signify the presence of collocation. Thus collocation extraction can be made possible by looking its linguistic aspect. In the present paper, an attempt has been made to make two different fuzzy sets of word combinations to be considered for collocations. Mutual information and t-test have been taken as basis for the construction of fuzzy sets. Two fuzzy set theoretical models have been proposed to identify collocations. It has been shown that fuzzy set theoretical approach works very well for collocation extraction. The working data has been based on a corpus of about one million words contained in different novels constituting project Gutenberg available on www.gutenberg.org.

Paraphrase Collocation Extraction Based on Binary Classification

以动宾关系的搭配为例研究复述搭配的抽取.具体地,该方法将复述搭配抽取视作二元分类问题,并综合使用了基于翻译、词典、极性词以及网络挖掘的多种特征.实验结果表明,所采用的二元分类方法对于抽取复述搭配是行之有效的,其中使用的各种特征对于提高复述搭配抽取的效果皆有帮助.利用该方法,共抽取出28万余对的复述搭配,其准确率超过70%.进一步的实验结果表明,使用抽取的复述搭配,可以为约40%的句子实现复述生成,从而说明了该方法的实际应用价值.;This paper addresses the problem of paraphrase collocation extraction by using “OBJ” relationship as a case study. Specifically, the proposed method recasts paraphrase collocation extraction as a binary classification problem, which combines multiple features based on translation, thesaurus, polarity words, and web mining. Experimental results show that the binary classification-based method is effective for paraphrase collocation extraction. Especially, the exploited features are all helpful for improving the extraction performance. With the proposed method, more than 280 000 pairs of paraphrase collocations are extracted, the precision of which is above 70%. Further experiments show that nearly 40% of sentences can be paraphrased by using the extracted paraphrase collocations, which demonstrates that the proposed method is useful in practice.

Lexical association measures and collocation extraction

We present an extensive empirical evaluation of collocation extraction methods based on lexical association measures and their combination. The experiments are performed on three sets of collocation candidates extracted from the Prague Dependency Treebank with manual morphosyntactic annotation and from the Czech National Corpus with automatically assigned lemmas and part-of-speech tags. The collocation candidates were manually labeled as collocational or non-collocational. The evaluation is based on measuring the quality of ranking the candidates according to their chance to form collocations. Performance of the methods is compared by precision-recall curves and mean average precision scores. The work is focused on two-word (bigram) collocations only. We experiment with bigrams extracted from sentence dependency structure as well as from surface word order. Further, we study the effect of corpus size on the performance of the individual methods and their combination.

Extending lexical association measures for collocation extraction

Collocations are linguistic phenomena that occur when two or more words appear together more often than by chance and whose meaning often cannot be inferred from the meanings of its parts. As collocations have found many applications in the fields of natural language processing, information retrieval, and text mining, extracting them from large corpora has been the focus of many studies over the past few years. In this paper, we introduce the notion of an extension pattern, a formalization of the idea of extending lexical association measures (AMs) defined for bigrams. An extension pattern provides a measure-independent way of extending AMs for extracting collocations of arbitrary length. We define different extension patterns and compare them on a task of extracting collocations from a newspaper corpus. We show that the stopword-sensitive extension patterns we propose outperform other extensions, which indicates that AMs could benefit by taking into account linguistic information about an n-gram’s part-of-speech pattern.

Introduction to Chinese Natural Language Processing

This book introduces Chinese language-processing issues and techniques to readers who already have a basic background in natural language processing (NLP). Since the major difference between Chinese and Western languages is at the word level, the book primarily focuses on Chinese morphological analysis and introduces the concept, structure, and interword semantics of Chinese words. The following topics are covered: a general introduction to Chinese NLP; Chinese characters, morphemes, and words and the characteristics of Chinese words that have to be considered in NLP applications; Chinese word segmentation; unknown word detection; word meaning and Chinese linguistic resources; interword semantics based on word collocation and NLP techniques for collocation extraction. Table of Contents: Introduction / Words in Chinese / Challenges in Chinese Morphological Processing / Chinese Word Segmentation / Unknown Word Identification / Word Meaning / Chinese Collocations / Automatic Chinese Collocation Extraction / Appendix / References / Author Biographies

Multilingual collocation extraction with a syntactic parser

An impressive amount of work was devoted over the past few decades to collocation extraction. The state of the art shows that there is a sustained interest in the morphosyntactic preprocessing of texts in order to better identify candidate expressions; however, the treatment performed is, in most cases, limited (lemmatization, POS-tagging, or shallow parsing). This article presents a collocation extraction system based on the full parsing of source corpora, which supports four languages: English, French, Spanish, and Italian. The performance of the system is compared against that of the standard mobile-window method. The evaluation experiment investigates several levels of the significance lists, uses a fine-grained annotation schema, and covers all the languages supported. Consistent results were obtained for these languages: parsing, even if imperfect, leads to a significant improvement in the quality of results, in terms of collocational precision (between 16.4 and 29.7%, depending on the language; 20.1% overall), MWE precision (between 19.9 and 35.8%; 26.1% overall), and grammatical precision (between 47.3 and 67.4%; 55.6% overall). This positive result bears a high importance, especially in the perspective of the subsequent integration of extraction results in other NLP applications.

Chunk Segmentation of Chinese Sentences Using a Combined Statistical and Rule-based Approach (CSRA)

Deep parsing of Chinese sentences is a very challenging task due to their complexity such as ambiguous word boundaries and meanings. An alternative mode of Chinese language processing is to perform shallow parsing of Chinese sentences in which chunk segmentation plays an important role. In this paper, we present a chunk segmentation algorithm using a combined statistical and rule-based approach (CSRA). The decision rules for refining chunk segmentation are generated from incorrectly segmented chunks from a statistical model which is built on a training corpus. Experimental results show that the CSRA works well and produces satisfactory chunk segmentation results for subsequent processes such as chunk tagging and chunk collocation extraction.

An evaluation of different statistical techniques of collocation extraction using a probability measure to word combinations

A collocation is a recurrent combination of words which appears more often than by chance. Collocations play an important role in many natural language processing tasks such as computational lexicography, word sense disambiguation, machine translation, information retrieval, etc. In this paper an attempt has been made to evaluate different collocation extraction techniques for a small corpus. For this purpose, some word pairs have been taken from the corpus and results of each technique have been shown and discussed. Finally, a probability measure has been suggested to filter out some free word combinations before applying these tests, which in turn improves the results obtained from the algorithms.

Comparison of Collocation Extraction Measures for Document Indexing

Automatic extraction of collocations from a corpus is a well-known problem in the field of natural language processing. It is typically carried out by employing some kind of a statistical measure that indicates whether or not two words occur together more often than by chance. As there is an abundance of these measures proposed by various authors, we have compared some of them on a task of extracting collocations from a corpus of Croatian legal documents for the purpose of document indexing. We propose and evaluate extensions of these measures for collocations consisting of three words

Using small random samples for the manual evaluation of statistical association measures

In this paper, we describe the empirical evaluation of statistical association measures for the extraction of lexical collocations from text corpora. We argue that the results of an evaluation experiment cannot easily be generalized to a different setting. Consequently, such experiments have to be carried out under conditions that are as similar as possible to the intended use of the measures. Finally, we show how an evaluation strategy based on random samples can reduce the amount of manual annotation work significantly, making it possible to perform many more evaluation experiments under specific conditions.

Gravity Counts for the boundaries of collocations

This paper compares several methods (MI,T-score, Dice) for the extraction of collocations and presents a new method called Gravity Counts. The respective methods are evaluated and compared, measuring the combinability and collocability for each pair of words within the moving span of three words in the corpus of “The Times” newspaper for the year 1995. The collocability of words is the basis for detection of the collocational chains, i.e. frequent recurrent uninterrupted strings of word-forms, with clear-cut boundaries, found in the corpus. Collocational chains obtained with the help of different methods are compared and their lexical, grammatical and semantic features discussed.

Using Parallel Corpora for Translation-Oriented Term Extraction

In many scientific, technological or political fields terminology and the production of up-to-date reference works is lagging behind, which causes problems to translators and results in inconsistent translations. Parallel corpora of texts already translated can be used as a resource for automatic extraction of terms and terminological collocations. Especially for smaller languages where existing resources are scarce, collecting and exploiting parallel corpora may be the chief method of obtaining terminological data. The paper describes how a methodology for multi-word term extraction and bilingual conceptual mapping was developed for Slovene-English terms. We used word-to-word alignment to extract a bilingual glossary of single-word terms, and for multi-word terms two methods were tested and compared. The statistical method is broadly applicable but gives results of very limited use, while the method of syntactic patterns extracts highly useful terminological phrases, however only from a tagged corpus. A vision of further development is given and how these methods might be incorporated into existing translation tools.

Automatic Extraction of Collocations From Korean Text

In this paper, we propose a statistical method to automaticallyextract collocations from Korean POS-tagged corpus. Since a large portion of language is represented by collocation patterns, the collocational knowledge provides a valuable resource for NLP applications. One difficulty of collocation extraction is that Korean has a partially free word order, which also appears in collocations. In this work, we exploit four statistics, ‘frequency’,‘randomness’, ‘convergence’, and ‘correlation' in order to take into account the flexible word order of Korean collocations. We separate meaningful bigrams using an evaluation function based on the four statistics and extend the bigrams to n-gram collocations using a fuzzy relation. Experiments show that this method works well for Korean collocations.

Extracting the Lowest-Frequency Words: Pitfalls and Possibilities

In a medical information extraction system, we use common word association techniques to extract side-effect-related terms. Many of these terms have a frequency of less than five. Standard word-association-based applications disregard the lowest-frequency words, and hence disregard useful information. We therefore devised an extraction system for the full word frequency range. This system computes the significance of association by the log-likelihood ratio and Fisher's exact test. The output of the system shows a recurrent, corpus-independent pattern in both recall and the number of significant words. We will explain these patterns by the statistical behavior of the lowest-frequency words. We used Dutch verb-particle combinations as a second and independent collocation extraction application to illustrate the generality of the observed phenomena. We will conclude that a) word-association-based extraction systems can be enhanced by also considering the lowest-frequency words, b) significance levels should not be fixed but adjusted for the optimal window size, c) hapax legomena, words occurring only once, should be disregarded a priori in the statistical analysis, and d) the distribution of the targets to extract should be considered in combination with the extraction method.

Aiding Web Searches by Statistical Classification Tools

We describe an infrastructure for the collection and management of large amounts of text, and discuss the possibility of information extraction and visualisation from text corpora with statistical methods. The paper gives an overview of processing steps, the contents of our text databases as well as different query facilities. Our focus is on the extraction and visualisation of collocations and their usage for aiding web searches.