Deep Language Processing Research Articles

A hybrid approach to Arabic named entity recognition

In this paper, we propose a hybrid named entity recognition (NER) approach that takes the advantages of rule-based and machine learning-based approaches in order to improve the overall system performance and overcome the knowledge elicitation bottleneck and the lack of resources for underdeveloped languages that require deep language processing, such as Arabic. The complexity of Arabic poses special challenges to researchers of Arabic NER, which is essential for both monolingual and multilingual applications. We used the hybrid approach to develop an Arabic NER system that is capable of recognizing 11 types of Arabic named entities: Person, Location, Organization, Date, Time, Price, Measurement, Percent, Phone Number, ISBN and File Name. Extensive experiments were conducted using decision trees, Support Vector Machines and logistic regression classifiers to evaluate the system performance. The empirical results indicate that the hybrid approach outperforms both the rule-based and the ML-based approaches when they are processed independently. More importantly, our system outperforms the state-of-the-art of Arabic NER in terms of accuracy when applied to ANERcorp standard dataset, with F-measures 0.94 for Person, 0.90 for Location and 0.88 for Organization.

Contextual question answering for the health domain

Studies have shown that natural language interfaces such as question answering and conversational systems allow information to be accessed and understood more easily by users who are unfamiliar with the nuances of the delivery mechanisms (e.g., keyword‐based search engines) or have limited literacy in certain domains (e.g., unable to comprehend health‐related content due to terminology barrier). In particular, the increasing use of the web for health information prompts us to reexamine our existing delivery mechanisms. We present enquireMe, which is a contextual question answering system that provides lay users with the ability to obtain responses about a wide range of health topics by vaguely expressing at the start and gradually refining their information needs over the course of an interaction session using natural language. enquireMe allows the users to engage in “conversations” about their health concerns, a process that can be therapeutic in itself. The system uses community‐driven question–answer pairs from the web together with a decay model to deliver the top scoring answers as responses to the users' unrestricted inputs. We evaluated enquireMe using benchmark data from WebMD and TREC to assess the accuracy of system‐generated answers. Despite the absence of complex knowledge acquisition and deep language processing, enquireMe is comparable to the state‐of‐the‐art question answering systems such as START as well as those interactive systems from TREC.

An electronic dictionary and translation system for Murrinh-Patha

This paper presents an electronic dictionary and translation system for the Australian language Murrinh-Patha. Its complex verbal structure makes learning Murrinh-Patha very difficult. Design learning materials or a dictionary which is easy to understand and to use also presents a challenge. This paper discusses some of the difficulties posed by the Murrinh-Patha verb system and proposes electronic resources which build on deep language processing to perform the required tasks.

Deep Processing of Korean and the Development of the Korean Resource Grammar

Kim, Jong-Bok, Yang, Jaehyung, Song, Sanghoun & Bond, Francis. 2011. Deep Processing of Korean and the Development of the Korean Resource Grammar. Linguistic Research 28(3), 635-672. The Korean Resource Grammar (KRG) is a broad-coverage, linguistically precise, and open-source grammar of Korean that has been developed since 2003. This paper reports how the KRG, aiming at deep processing of Korean, has gone through two main developing phases, in particular from a linguistically-oriented deep grammar to a wider-coverage with grammar customization. The development of the KRG in these two phases, also participating in the international collaboration project DELPH-IN (Deep Language Processing in HPSG), has resulted in promising directions for the improvement of deep parsing as well as generation methods in a more systematic way so that it can be geared toward practical NLP applications such as machine translation.

Construction of Thai Lexicon from Existing Dictionaries and Texts on the Web

A lexicon is an important linguistic resource needed for both shallow and deep language processing. Currently, there are few machine-readable Thai dictionaries available, and most of them do not satisfy the computational requirements. This paper presents the design of a Thai lexicon named the TCL's Computational Lexicon (TCLLEX) and proposes a method to construct a large-scale Thai lexicon by re-using two existing dictionaries and a large number of texts on the Internet. In addition to morphological, syntactic, semantic case role and logical information in the existing dictionaries, a sort of semantic constraint called selectional preference is automatically acquired by analyzing Thai texts on the web and then added into the lexicon. In the acquisition process of the selectional preferences, the so-called Bayesian Information Criterion (BIC) is applied as the measure in a tree cut model. The experiments are done to verify the feasibility and effectiveness of obtained selection preferences.