Natural Language Information Retrieval Research Articles

Semantic concept model using Wikipedia semantic features

Wikipedia has become a high coverage knowledge source which has been used in many research areas such as natural language processing, text mining and information retrieval. Several methods have been introduced for extracting explicit or implicit relations from Wikipedia to represent semantics of concepts/words. However, the main challenge in semantic representation is how to incorporate different types of semantic relations to capture more semantic evidences of the associations of concepts. In this article, we propose a semantic concept model that incorporates different types of semantic features extracting from Wikipedia. For each concept that corresponds to an article, four semantic features are introduced: template links, categories, salient concepts and topics. The proposed model is based on the probability distributions that are defined for these semantic features of a Wikipedia concept. The template links and categories are the document-level features which are directly extracted from the structured information included in the article. On the other hand, the salient concepts and topics are corpus-level features which are extracted to capture implicit relations among concepts. For the salient concepts feature, the distributional-based method is utilised on the hypertext corpus to extract this feature for each Wikipedia concept. Then, the probability product kernel is used to improve the weight of each concept in this feature. For the topic feature, the Labelled latent Dirichlet allocation is adapted on the supervised multi-label of Wikipedia to train the probabilistic model of this feature. Finally, we used the linear interpolation for incorporating these semantic features into the probabilistic model to estimate the semantic relation probability of the specific concept over Wikipedia articles. The proposed model is evaluated on 12 benchmark datasets in three natural language processing tasks: measuring the semantic relatedness of concepts/words in general and in the biomedical domain, semantic textual relatedness measurement and measuring the semantic compositionality of noun compounds. The model is also compared with five methods that depends on separate semantic features in Wikipedia. Experimental results show that the proposed model achieves promising results in three tasks and outperforms the baseline methods in most of the evaluation datasets. This implies that incorporation of explicit and implicit semantic features is useful for representing semantics of concepts in Wikipedia.

The Design of Lexical Database for Indonesian Language

Kamus Besar Bahasa Indonesia (KBBI), an official dictionary for Indonesian language, provides lists of words with their meaning. The online version can be accessed via Internet network. Another online dictionary is Kateglo. KBBI online and Kateglo only provides an interface for human. A machine cannot retrieve data from the dictionary easily without using advanced techniques. Whereas, lexical of words is required in research or application development which related to natural language processing, text mining, information retrieval or sentiment analysis. To address this requirement, we need to build a lexical database which provides well-defined structured information about words. A well-known lexical database is WordNet, which provides the relation among words in English. This paper proposes the design of a lexical database for Indonesian language based on the combination of KBBI 4th edition, Kateglo and WordNet structure. Knowledge representation by utilizing semantic networks depict the relation among words and provide the new structure of lexical database for Indonesian language. The result of this design can be used as the foundation to build the lexical database for Indonesian language.

Report on the Joint Workshop on Bibliometric-enhanced Information Retrieval and Natural Language Processing for Digital Libraries (BIRNDL 2016)

The large scale of scholarly publications poses a challenge for scholars in information seeking and sensemaking. Bibliometric, information retrieval (IR), text mining, and NLP techniques could help in these activities, but are not yet widely used in digital libraries. The BIRNDL workshop was held at the Joint Conference on Digital Libraries (JCDL 2016) in Newark, NJ. It intended to stimulate IR researchers and digital library professionals to elaborate on new approaches in natural language processing, information retrieval, scientometric, and recommendation techniques that can advance the state-of-the-art in scholarly document understanding, analysis, and retrieval at scale. The workshop incorporated three paper sessions and the 2nd edition of the CL-SciSumm Shared Task.

An ensemble scheme based on language function analysis and feature engineering for text genre classification

Text genre classification is the process of identifying functional characteristics of text documents. The immense quantity of text documents available on the web can be properly filtered, organised and retrieved with the use of text genre classification, which may have potential use on several other tasks of natural language processing and information retrieval. Genre may refer to several aspects of text documents, such as function and purpose. The language function analysis (LFA) concentrates on single aspect of genres and it aims to classify text documents into three abstract classes, such as expressive, appellative and informative. Text genre classification is typically performed by supervised machine learning algorithms. The extraction of an efficient feature set to represent text documents is an essential task for building a robust classification scheme with high predictive performance. In addition, ensemble learning, which combines the outputs of individual classifiers to obtain a robust classification scheme, is a promising research field in machine learning research. In this regard, this article presents an extensive comparative analysis of different feature engineering schemes (such as features used in authorship attribution, linguistic features, character n-grams, part of speech n-grams and the frequency of the most discriminative words) and five different base learners (Naïve Bayes, support vector machines, logistic regression, k-nearest neighbour and Random Forest) in conjunction with ensemble learning methods (such as Boosting, Bagging and Random Subspace). Based on the empirical analysis, an ensemble classification scheme is presented, which integrates Random Subspace ensemble of Random Forest with four types of features (features used in authorship attribution, character n-grams, part of speech n-grams and the frequency of the most discriminative words). For LFA corpus, the highest average predictive performance obtained by the proposed scheme is 94.43%.

QAPD: an ontology-based question answering system in the physics domain

The tremendous development in information technology led to an explosion of data and motivated the need for powerful yet efficient strategies for knowledge discovery. Question answering (QA) systems made it possible to ask questions and retrieve answers using natural language queries. In ontology-based QA system, the knowledge-based data, where the answers are sought, have a structured organization. The question-answer retrieval of ontology knowledge base provides a convenient way to obtain knowledge for use. In this paper, QAPD, an ontology-based QA system applied to the physics domain, which integrates natural language processing, ontologies and information retrieval technologies to provide informative information for users, is presented. This system allows users to retrieve information from formal ontologies using input queries formulated in natural language. We proposed inferring schema mapping method, which uses the combination of semantic and syntactic information, and attribute-based inference to transform users' questions into ontological knowledge base query. In addition, a novel domain ontology for physics domain, called EAEONT, is presented. Relevant standards and regulations have been utilized extensively during the ontology building process. The original characteristic of system is the strategy used to fill the gap between users' expressiveness and formal knowledge representation. This system has been developed and tested on the English language and using an ontology modeling the physics domain. The performance level achieved enables the use of the system in real environments.

A novel robust Arabic light stemmer

The stemming is the process of transforming a word into its root or stem, hence, it is considered as a crucial pre-processing step before tackling any task of natural language processing or information retrieval. However, in the case of Arabic language, finding an effective stemming algorithm seems to be quite difficult, since the Arabic language has a specific morphology, which is different from many other languages. Although, there exist several algorithms in literature addressing the Arabic stemming issue, unfortunately, most of them are restricted to a limited number of words, present some confusions between original letters and affixes, and usually employ dictionary of words or patterns. For that purpose, we propose the design and implementation of a novel Arabic light stemmer, which is based on some new rules for stripping prefixes, suffixes and infixes in a smart way. And in our knowledge, it is the first work dealing with Arabic infixes with regards to their irregular rules. The empirical evaluation was conducted on a new Arabic data-set (called ARASTEM), which was conceived and collected from several Arabic discussion forums containing dialectical Arabic and modern pseudo-Arabic languages. Hence, we present a comparative investigation between our new stemmer and other existing stemmers using Paice’s parameters, namely: Under Stemming Index (UI), Over Stemming Index (OI) and Stemming Weight (SW). Results show that the proposed Arabic light stemmer maintains consistently high performances and outperforms several existing light stemmers.

Building spatial temporal relation graph of concepts pair using web repository

Mining semantic relations between concepts underlies many fundamental tasks including natural language processing, web mining, information retrieval, and web search. In order to describe the semantic relation between concepts, in this paper, the problem of automatically generating spatial temporal relation graph (STRG) of semantic relation between concepts is studied. The spatial temporal relation graph of semantic relation between concepts includes relation words, relation sentences, relation factor, relation graph, faceted feature, temporal feature, and spatial feature. The proposed method can automatically generate the spatial temporal relation graph (STRG) of semantic relation between concepts, which is different from the manually generated annotation repository such as WordNet and Wikipedia. Moreover, the proposed method does not need any prior knowledge such as ontology or the hierarchical knowledge base such as WordNet. Empirical experiments on real dataset show that the proposed algorithm is effective and accurate.

Normalizing acronyms and abbreviations to aid patient understanding of clinical texts: ShARe/CLEF eHealth Challenge 2013, Task 2.

BackgroundThe ShARe/CLEF eHealth challenge lab aims to stimulate development of natural language processing and information retrieval technologies to aid patients in understanding their clinical reports. In clinical text, acronyms and abbreviations, also referenced as short forms, can be difficult for patients to understand. For one of three shared tasks in 2013 (Task 2), we generated a reference standard of clinical short forms normalized to the Unified Medical Language System. This reference standard can be used to improve patient understanding by linking to web sources with lay descriptions of annotated short forms or by substituting short forms with a more simplified, lay term.MethodsIn this study, we evaluate 1) accuracy of participating systems’ normalizing short forms compared to a majority sense baseline approach, 2) performance of participants’ systems for short forms with variable majority sense distributions, and 3) report the accuracy of participating systems’ normalizing shared normalized concepts between the test set and the Consumer Health Vocabulary, a vocabulary of lay medical terms.ResultsThe best systems submitted by the five participating teams performed with accuracies ranging from 43 to 72 %. A majority sense baseline approach achieved the second best performance. The performance of participating systems for normalizing short forms with two or more senses with low ambiguity (majority sense greater than 80 %) ranged from 52 to 78 % accuracy, with two or more senses with moderate ambiguity (majority sense between 50 and 80 %) ranged from 23 to 57 % accuracy, and with two or more senses with high ambiguity (majority sense less than 50 %) ranged from 2 to 45 % accuracy. With respect to the ShARe test set, 69 % of short form annotations contained common concept unique identifiers with the Consumer Health Vocabulary. For these 2594 possible annotations, the performance of participating systems ranged from 50 to 75 % accuracy.ConclusionShort form normalization continues to be a challenging problem. Short form normalization systems perform with moderate to reasonable accuracies. The Consumer Health Vocabulary could enrich its knowledge base with missed concept unique identifiers from the ShARe test set to further support patient understanding of unfamiliar medical terms.

Multi-granularity sequence labeling model for acronym expansion identification

Identifying expansion forms for acronyms is beneficial to many natural language processing and information retrieval tasks. In this work, we study the problem of finding expansions in texts for given acronym queries by modeling the problem as a sequence labeling task. However, it is challenging for traditional sequence labeling models like Conditional Random Fields (CRF) due to the complexity of the input sentences and the substructure of the categories. In this paper, we propose a Latent-state Neural Conditional Random Fields model (LNCRF) to deal with the challenges. On one hand, we extend CRF by coupling it with nonlinear hidden layers to learn multi-granularity hierarchical representations of the input data under the framework of Conditional Random Fields. On the other hand, we introduce latent variables to capture the fine granular information from the intrinsic substructures within the structured output labels implicitly. The experimental results on real data show that our model achieves the best performance against the state-of-the-art baselines.

Framework for Improved Question Answering System

Background/Objectives: The proposed frame work uses statistical based similarity measure for QAS, which gives short and exact answer to user query. The entire frame work is focusing similarity computation between question and answer pair. Methods/Statistical Analysis: The involvement of similarity computation between question and answer pair should be investigated in this work QAS. Statistical based similarity measure is an important issue as compared to similarity computation in many fields such as, Natural Language Processing, Information Retrieval, Ontology Mapping, Knowledge Acquisition and Question Answering System. Semantic web has been used to present the structure knowledge representation between questions and answer keywords. Ontology can be used as important proposal to enable semantic similarity between keywords present in the question and answer pairs. Findings: The proposed frame work investigate, how the statistical based similarity measure used in these system helps to improve the performance of the QA system for all Wh-questions. We find the keywords having most similar meaning will return the answer as the final answer. The best performance is achieved by extracting the relevant snippets information form Google search engine. The performance of the system will change as the retrieval of the document is increases beyond 15; this could indicate that restriction in the retrieval of document is helps to optimize the performance of the system. Application/Improvements: This frame work is applicable to all type of search engines which helps to finds most relevant answer to user questions. In future, we implement our frame work for questions having more than five keywords.

FEATURE SELECTION USING SINGULAR VALUE DECOMPOSITION AND ORTHOGONAL CENTROID FEATURE SELECTION FOR TEXT CLASSIFICATION

Text mining is a narrow research field of data mining, which focuses on discovering new information from text document collections, mainly by using techniques from data mining, machine learning, natural language processing and information retrieval. Text classification is the process of analyzing text content and then giving decision whether this text can belong to one group, many groups or it does not belong to the text group which is defined before. On over the world, there have been many effective researches on this problem, especially on texts in English. However, there have been few researches on Vietnamese texts. Moreover, these researching results and applications are still limited partly due to the typical characteristics of Vietnamese language in term of words and sentences and there are many words with many meanings in many different contexts. Text classification problem is the one with many featues, thus to improve the effectiveness of text classification is the aim of may researchers. In this research, the author constructs two methods of feature selection: singular value decomposition and optimal orthogonal centroid feature selection in text classification with high efficiency of calculation proven on English text document and now they are proven on Vietnamese text document. There are many classification techniques, but we implemented on the learning machine algorithms support vector machines. This method has been proven to be effective for text classification problems. With the technique of feature selection singular value decomposition and optimal orthogonal centroid feature selection, the implementing result higher than that of traditional method.

Automatic Wordnet Development for Low-Resource Languages using Cross-Lingual WSD

‎Wordnets are an effective resource for natural language processing and information retrieval‎, ‎especially for semantic processing and meaning related tasks‎. ‎So far‎, ‎wordnets have been constructed for many languages‎. ‎However‎, ‎the automatic development of wordnets for low-resource languages has not been well studied‎. ‎In this paper‎, ‎an Expectation-Maximization algorithm is used to create high quality and large scale wordnets for poor-resource languages‎. ‎The proposed method benefits from possessing cross-lingual word sense disambiguation and develops a wordnet by only using a bi-lingual dictionary and a mono-lingual corpus‎. ‎The proposed method has been executed with Persian language and the resulting wordnet has been evaluated through several experiments‎. ‎The results show that the induced wordnet has a precision score of 90% and a recall score of 35%‎.

Automatic wordnet development for low-resource languages using cross-lingual WSD

Wordnets are an effective resource for natural language processing and information retrieval, especially for semantic processing and meaning related tasks. So far, wordnets have been constructed fo...

Large-Scale Text Similarity Computing with Spark

Text understanding is a hot research in Natural Language Processing and Information Retrieval. In recent years, it has received wide attention and research. In the era of big data, Understanding text in large-scale datasets is a challenge. Although the earliest systems designed for these workloads, such as MapReduce, gave users a powerful, but low-level, procedural programming interface. So, MapReduce doesn’t compose well for lager text applications. Recently, Spark, an in-memory cluster-computing platform, has been proposed. It has emerged as a popular framework for large-scale data processing and analytics. It provides a general-purpose efficient cluster computing engine and simpler for the end users. In this work, we consider using Vector Space Model (VSM) and TF-IDF weighting schema and feature hashing feature extraction techniques in order to solve the problem of large-scale text data similarity computing by Spark. As a result, Experimental results that using Spark in order to solve document similarity computation problems as soon as quickly by 20Newsgroups. In additions, It is more benefit from document classification and clustering of machine learning tasks.

Improving a Lightweight Stemmer for Gujarati Language

The origin of route of text mining is the process of stemming. It is usually used in several types of applications such as Natural Language Processing (NLP), Information Retrieval (IR) and Text Mining (TM) including Text Categorization (TC), Text Summarization (TS). Establish a stemmer effective for the language of Gujarati has been always a search domain hot since the Gujarati has a very different structure and difficult that the other language due to the rich morphology.

Ensemble of keyword extraction methods and classifiers in text classification

Automatic keyword extraction is an important research direction in text mining, natural language processing and information retrieval. Keyword extraction enables us to represent text documents in a condensed way. The compact representation of documents can be helpful in several applications, such as automatic indexing, automatic summarization, automatic classification, clustering and filtering. For instance, text classification is a domain with high dimensional feature space challenge. Hence, extracting the most important/relevant words about the content of the document and using these keywords as the features can be extremely useful. In this regard, this study examines the predictive performance of five statistical keyword extraction methods (most frequent measure based keyword extraction, term frequency-inverse sentence frequency based keyword extraction, co-occurrence statistical information based keyword extraction, eccentricity-based keyword extraction and TextRank algorithm) on classification algorithms and ensemble methods for scientific text document classification (categorization). In the study, a comprehensive study of comparing base learning algorithms (Naïve Bayes, support vector machines, logistic regression and Random Forest) with five widely utilized ensemble methods (AdaBoost, Bagging, Dagging, Random Subspace and Majority Voting) is conducted. To the best of our knowledge, this is the first empirical analysis, which evaluates the effectiveness of statistical keyword extraction methods in conjunction with ensemble learning algorithms. The classification schemes are compared in terms of classification accuracy, F-measure and area under curve values. To validate the empirical analysis, two-way ANOVA test is employed. The experimental analysis indicates that Bagging ensemble of Random Forest with the most-frequent based keyword extraction method yields promising results for text classification. For ACM document collection, the highest average predictive performance (93.80%) is obtained with the utilization of the most frequent based keyword extraction method with Bagging ensemble of Random Forest algorithm. In general, Bagging and Random Subspace ensembles of Random Forest yield promising results. The empirical analysis indicates that the utilization of keyword-based representation of text documents in conjunction with ensemble learning can enhance the predictive performance and scalability of text classification schemes, which is of practical importance in the application fields of text classification.

Multi-Class Opinion Classification for Vietnamese Hotel Reviews

Opinion mining or sentiment analysis is a new research field. However, it plays an important role in computer sciences, and attracts interests of researchers, companies, as well as organizations. As afield of research, it is closely related to natural language processing, machine learning, text mining, and information retrieval. In this paper, the main method we use to solve the issues of classifying Vietnamese reviews in hospitality is machine learning techniques. The results obtained from the experimental phase are optimistic.

Improving a Lightweight Stemmer for Gujarati Language

The origin of route of text mining is the process of stemming. It is usually used in several types of applications such as Natural Language Processing (NLP), Information Retrieval (IR) and Text Mining (TM) including Text Categorization (TC), Text Summarization (TS). Establish a stemmer effective for the language of Gujarati has been always a search domain hot since the Gujarati has a very different structure and difficult that the other language due to the rich morphology.

Constructing Automated Scoring Model for Human Translation with Multidisciplinary Technologies

This study constructed a computer scoring model for Chinese EFL learners’ English-to-Chinese translations using multidisciplinary techniques in corpus linguistics, natural language processing, information retrieval and statistics. The proposed model, once implemented as computer software, can score English-to-Chinese translations in large-scale examinations. This study built five tentative scoring models with 50, 100, 130, 150 and 180 translations as the training set for 300 translations of an expository writing. The correlation coefficients between the computed scores of these models and human-assigned scores were above 0.8. The results further indicated that the computed scores with 130 training translations were closest to human-assigned scores. Therefore, it was concluded that the text features extracted in this research were effective and the finalized model can produce reliable scores for Chinese EFL learners’ English-to-Chinese expository translations.

Information extraction for personalised services based on conference alerts

Text mining is moderately new research area at the interaction of data mining, natural language processing (NLP), machine learning and information retrieval. The interconnected task, information extraction is a text transforming that places a specified set of significant items in a natural-language document. It distils organised data or knowledge from unstructured text by recognising references to named entities and additionally expressed relationships between such entities. We present a new schema for text mining as information extraction for prediction, which uses a learn information extraction system to transform text into more structures data which is then be further analysed or mine for discovering more general patterns and interesting relationships. This paper presents the work obtained by applying information extraction (IE) technique to a corpus of conference announcement posted on conference web newsgroups. The work is analysis of extracted essential name entities that were used to find the patterns of recent trends in research area and it also provide a platform to explore more on NLP aspects.