Unsupervised Morphological Segmentation Research Articles

Usable Amharic text corpus for natural language processing applications

In this paper, we describe the preparation of a usable Amharic text corpus for different Natural Language Processing (NLP) applications. Natural language applications, such as document classification, topic modeling, machine translation, speech recognition, and others, suffer greatly from a lack of digital resources. This is especially true for Amharic, a resource-constrained, morphologically rich, and complex language. In response to this, a total of 67,739 Amharic news documents consisting of 8 different categories from online sources are collected. The collected corpus passes through a number of pre-processing steps including; data cleaning, text normalization and punctuation correction. To validate the usability of the collected corpora from different domains, a baseline document classification experiment was conducted. Experimental results show that, 84.53% accuracy is registered using deep learning in the absence of linguistic information. Finding indicated that it is possible to use the prepared corpora for different natural language applications in the absence of linguistic resources such as stemmer and dictionary despite the complexity of Amharic language. We are further working towards Amharic news document classification by incorporating a linguistic independent stop-word detection, stemming and unsupervised morphological segmentation of Amharic documents.

CT-ORG, a new dataset for multiple organ segmentation in computed tomography

Despite the relative ease of locating organs in the human body, automated organ segmentation has been hindered by the scarcity of labeled training data. Due to the tedium of labeling organ boundaries, most datasets are limited to either a small number of cases or a single organ. Furthermore, many are restricted to specific imaging conditions unrepresentative of clinical practice. To address this need, we developed a diverse dataset of 140 CT scans containing six organ classes: liver, lungs, bladder, kidney, bones and brain. For the lungs and bones, we expedited annotation using unsupervised morphological segmentation algorithms, which were accelerated by 3D Fourier transforms. Demonstrating the utility of the data, we trained a deep neural network which requires only 4.3 s to simultaneously segment all the organs in a case. We also show how to efficiently augment the data to improve model generalization, providing a GPU library for doing so. We hope this dataset and code, available through TCIA, will be useful for training and evaluating organ segmentation models.

Script Independent Morphological Segmentation for Arabic Maghrebi Dialects: An Application to Machine Translation

This research deals with resources creation for under-resourced languages. We try to adapt existing resources for other resourced-languages to process less resourced ones. We focus on Arabic dialects of the Maghreb, namely Algerian, Moroccan and Tunisian. We first adapt a well known statistical word segmenter to segment Algerian dialect texts written in both Arabic and Latin scripts. We demonstrate that unsupervised morphological segmentation could be applied to Arabic dialects regardless of used script. Next, we use this kind of segmentation to improve statistical machine translation scores between the tree Maghrebi dialects and French. We use a parallel multidialectal corpus that includes six Arabic dialects in addition to MSA and French. We achieved interesting results. Regardsto word segmentation, the rate of correctly segmented words reached 70% for those written in Latin scriptand 79% for those written in Arabic script. For machine translation, the unsupervised morphological segmentation helped to decrease out of vocabulary words rates by a minimum of 35%.

Roughness-based landscape segmentation via fuzzy clustering: potentials of a robust surface texture estimator

In this study, the impact of robust surface texture indices on the unsupervised morphological segmentation of an alpine basin is explored. Roughness indices calculated on a high resolution digital terrain model derived by means of airborne LiDAR (Light Detection and Ranging) are used as input features in the clustering procedure. The segmentation in textural classes is based on a fuzzy clustering approach, permitting the optimal selection of the number of clusters and fuzziness of the classification. A comparison is made between clusters derived from variogram-based indices and MAD–based (median absolute differences) roughness indices within a small alpine basin. As expected, with both approaches, the fuzzy clustering revealed a high fuzziness and a high degree of mixing between textural classes. However, the segmentation derived using MAD-based roughness indices shows relevant improvements; in particular, the spatial patterns of the MAD-based classification show less artifacts respect to the variogram-based one.

Tree Structured Dirichlet Processes for Hierarchical Morphological Segmentation

This article presents a probabilistic hierarchical clustering model for morphological segmentation. In contrast to existing approaches to morphology learning, our method allows learning hierarchical organization of word morphology as a collection of tree structured paradigms. The model is fully unsupervised and based on the hierarchical Dirichlet process. Tree hierarchies are learned along with the corresponding morphological paradigms simultaneously. Our model is evaluated on Morpho Challenge and shows competitive performance when compared to state-of-the-art unsupervised morphological segmentation systems. Although we apply this model for morphological segmentation, the model itself can also be used for hierarchical clustering of other types of data.

Unsupervised morphological segmentation of tissue compartments in histopathological images.

Algorithmic segmentation of histologically relevant regions of tissues in digitized histopathological images is a critical step towards computer-assisted diagnosis and analysis. For example, automatic identification of epithelial and stromal tissues in images is important for spatial localisation and guidance in the analysis and characterisation of tumour micro-environment. Current segmentation approaches are based on supervised methods, which require extensive training data from high quality, manually annotated images. This is often difficult and costly to obtain. This paper presents an alternative data-independent framework based on unsupervised segmentation of oropharyngeal cancer tissue micro-arrays (TMAs). An automated segmentation algorithm based on mathematical morphology is first applied to light microscopy images stained with haematoxylin and eosin. This partitions the image into multiple binary ‘virtual-cells’, each enclosing a potential ‘nucleus’ (dark basins in the haematoxylin absorbance image). Colour and morphology measurements obtained from these virtual-cells as well as their enclosed nuclei are input into an advanced unsupervised learning model for the identification of epithelium and stromal tissues. Here we exploit two Consensus Clustering (CC) algorithms for the unsupervised recognition of tissue compartments, that consider the consensual opinion of a group of individual clustering algorithms. Unlike most unsupervised segmentation analyses, which depend on a single clustering method, the CC learning models allow for more robust and stable detection of tissue regions. The proposed framework performance has been evaluated on fifty-five hand-annotated tissue images of oropharyngeal tissues. Qualitative and quantitative results of the proposed segmentation algorithm compare favourably with eight popular tissue segmentation strategies. Furthermore, the unsupervised results obtained here outperform those obtained with individual clustering algorithms.

Unsupervised morphological segmentation based on affixality measurements

In this paper, we present a method for unsupervised morphological segmentation for multi-slot morphology based on affixality measurements. These measurements quantify three linguistic characteristics of affixes: (1) they combine with many low frequency word-bases (high combinatorial capacity), (2) although they are relatively few, they help to maximize the size of a lexicon (economy principle), i.e. speakers know more words by remembering fewer morphological items, and (3) they are very frequent, so they contain less information than word-bases (entropy), i.e. borders between affixes and stems can be detected by finding entropy peaks. Several experiments combining these measurements were conducted to find the best way to apply them to data. The best strategy consists in successive segmentation when the average of the affixality measurements surpasses a threshold of 0.5. Also, we compared this strategy with some state-of-the-art methods for unsupervised morphological segmentation (Morfessor and ParaMor). Our method outperformed these methods, when tested in a hand-made corpus. Results indicate that our proposal is competitive at least for the morphological segmentation of Spanish words.

Automatically identifying blend splinters that are morpheme candidates

Forms such as -topia in privatopia or -ercise in dancercise are known as blend splinters: they might not be morphemes, but they are clearly involved in word formation. This article offers an automated method that can highlight blend splinters which have the potential to become morphemes in their own right. For instance, the word alcoholic has given rise a large number of blends such as workaholic or rageaholic , so that the splinter -holic is now recognized as a morpheme in the Oxford English Dictionary Online. Because of the sheer number of newly coined blends, it is difficult to identify splinters that are turning into morphemes on the sole basis of human observation. It would therefore be desirable to have an automated method that could process large amounts of data and identify such elements. This article develops such a method, relying on unsupervised morphological segmentation (Harris, 1955). A custom blend database was established for this purpose. The method is able to detect splinters mentioned in previous research, such as -tainment , -ercise , and cyber- , but in addition, it also detects elements that have not been discussed so far, including -tastic , -sumer , and -verse .

Computational strategies for reducing annotation effort in language documentation

With the urgent need to document the world's dying languages, it is important to explore ways to speed up language documentation efforts. One promising avenue is to use techniques from computational linguistics to automate some of the process. Here we consider unsupervised morphological segmentation and active learning for creating interlinear glossed text (IGT) for the Mayan language Uspanteko. The practical goal is to produce a totally annotated corpus that is as accurate as possible given limited time for manual annotation. We discuss results from several experiments that suggest there is indeed much promise in these methods but also show that further development is necessary to make them robustly useful for a wide range of conditions and tasks. We also provide a detailed discussion of how two documentary linguists perceived machine support in IGT production and how their annotation performance varied with different levels of machine support.