TermeX: A Tool for Collocation Extraction

Abstract

Collocations – word combinations occurring together more often than by chance – have a wide range of NLP applications. Many approaches for automating collocation extraction based on lexical association measures have been proposed in the literature. This paper presents TermeX – a tool for efficient extraction of collocations based on a variety of association measures. TermeX implements POS filtering and lemmatization, and is capable of extracting collocations up to length four. We address trade-offs between high memory consumption and processing speed and propose an efficient implementation. Our implementation allows for processing time linear to corpus size and memory consumption linear to the number of word types.KeywordsMachine TranslationHash TableVector VersusAssociation MeasureNatural Language GenerationThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.