Standard Corpus Research Articles

Special Issue on the Curative Power of Medical Data

With the massive amounts of medical data made available online, language technologies have proven to be indispensable in processing biomedical and molecular biology literature, health data or patient records. With huge amount of reports, evaluating their impact has long ceased to be a trivial task. Linking the contents of these documents to each other, as well as to specialized ontologies, could enable access to and the discovery of structured clinical information and could foster a major leap in natural language processing and in health research. The aim of this Special Issue, “Curative Power of Medical Data” in Data, is to gather innovative approaches for the exploitation of biomedical data using semantic web technologies and linked data by developing a community involvement in biomedical research. This Special Issue contains four surveys, which include a wide range of topics, from the analysis of biomedical articles writing style, to automatically generating tests from medical references, constructing a Gold standard biomedical corpus or the visualization of biomedical data.

Arabic named entity recognition using deep learning approach

Most of the Arabic Named Entity Recognition (NER) systems depend massively on external resources and handmade feature engineering to achieve state-of-the-art results. To overcome such limitations, we proposed, in this paper, to use deep learning approach to tackle the Arabic NER task. We introduced a neural network architecture based on bidirectional Long Short-Term Memory (LSTM) and Conditional Random Fields (CRF) and experimented with various commonly used hyperparameters to assess their effect on the overall performance of our system. Our model gets two sources of information about words as input: pre-trained word embeddings and character-based representations and eliminated the need for any task-specific knowledge or feature engineering. We obtained state-of-the-art result on the standard ANERcorp corpus with an F1 score of 90.6%.

Towards the Automatic Classification of Speech Subjects in the Danish Parliament Corpus

This paper addresses the semi-automatic subject area annotation of the Danish Parliament Corpus 2009-2017 in order to construct a gold standard corpus for automatic classification. The corpus consists of the transcriptions of the speeches in the Danish parliamentary meetings. In our annotation work, we mainly use subject categories proposed by Danish scholars in political sciences. The relevant subjects areas of the speeches have been manually annotated using the titles of the agendas items for the parliamentary meetings and then the subjects areas have been assigned to the corresponding speeches. Some subjects co-occur in the agendas, since they are often debated at the same time. The fact that the same speech can belong to more subject areas is further analysed. Currently, more than 29,000 speeches have been classified using the titles of the agenda items. Different evaluation strategies have been applied. We also describe automatic classification experiments on a subset of the corpus using feature extracted with NLP techniques. The best results (96% F-score) were obtained using features extracted from the agenda items. These results indicate that the gold standard corpus and agenda items can be used for automatically classify parliamentary debates with high accuracy.

COPIOUS: A gold standard corpus of named entities towards extracting species occurrence from biodiversity literature.

Background Species occurrence records are very important in the biodiversity domain. While several available corpora contain only annotations of species names or habitats and geographical locations, there is no consolidated corpus that covers all types of entities necessary for extracting species occurrence from biodiversity literature. In order to alleviate this issue, we have constructed the COPIOUS corpus—a gold standard corpus that covers a wide range of biodiversity entities. Results Two annotators manually annotated the corpus with five categories of entities, i.e. taxon names, geographical locations, habitats, temporal expressions and person names. The overall inter-annotator agreement on 200 doubly-annotated documents is approximately 81.86% F-score. Amongst the five categories, the agreement on habitat entities was the lowest, indicating that this type of entity is complex. The COPIOUS corpus consists of 668 documents downloaded from the Biodiversity Heritage Library with over 26K sentences and more than 28K entities. Named entity recognisers trained on the corpus could achieve an F-score of 74.58%. Moreover, in recognising taxon names, our model performed better than two available tools in the biodiversity domain, namely the SPECIES tagger and the Global Name Recognition and Discovery. More than 1,600 binary relations of Taxon-Habitat, Taxon-Person, Taxon-Geographical locations and Taxon-Temporal expressions were identified by applying a pattern-based relation extraction system to the gold standard. Based on the extracted relations, we can produce a knowledge repository of species occurrences. Conclusion The paper describes in detail the construction of a gold standard named entity corpus for the biodiversity domain. An investigation of the performance of named entity recognition (NER) tools trained on the gold standard revealed that the corpus is sufficiently reliable and sizeable for both training and evaluation purposes. The corpus can be further used for relation extraction to locate species occurrences in literature—a useful task for monitoring species distribution and preserving the biodiversity.

Web document summarization by exploiting social context with matrix co-factorization

In the context of social media, users usually post relevant information corresponding to the contents of events mentioned in a Web document. This information posses two important values in that (i) it reflects the content of an event and (ii) it shares hidden topics with sentences in the main document. In this paper, we present a novel model to capture the nature of relationships between document sentences and post information (comments or tweets) in sharing hidden topics for summarization of Web documents by utilizing relevant post information. Unlike previous methods which are usually based on hand-crafted features, our approach ranks document sentences and user posts based on their importance to the topics. The sentence-user-post relation is formulated in a share topic matrix, which presents their mutual reinforcement support. Our proposed matrix co-factorization algorithm computes the score of each document sentence and user post and extracts the top ranked document sentences and comments (or tweets) as a summary. We apply the model to the task of summarization on three datasets in two languages, English and Vietnamese, of social context summarization and also on DUC 2004 (a standard corpus of the traditional summarization task). According to the experimental results, our model significantly outperforms the basic matrix factorization and achieves competitive ROUGE-scores with state-of-the-art methods.

Ontology-based approach to enhance medical web information extraction

PurposeThe purpose of this study is to propose a framework for extracting medical information from the Web using domain ontologies. Patient–Doctor conversations have become prevalent on the Web. For instance, solutions like HealthTap or AskTheDoctors allow patients to ask doctors health-related questions. However, most online health-care consumers still struggle to express their questions efficiently due mainly to the expert/layman language and knowledge discrepancy. Extracting information from these layman descriptions, which typically lack expert terminology, is challenging. This hinders the efficiency of the underlying applications such as information retrieval. Herein, an ontology-driven approach is proposed, which aims at extracting information from such sparse descriptions using a meta-model.Design/methodology/approachA meta-model is designed to bridge the gap between the vocabulary of the medical experts and the consumers of the health services. The meta-model is mapped with SNOMED-CT to access the comprehensive medical vocabulary, as well as with WordNet to improve the coverage of layman terms during information extraction. To assess the potential of the approach, an information extraction prototype based on syntactical patterns is implemented.FindingsThe evaluation of the approach on the gold standard corpus defined in Task1 of ShARe CLEF 2013 showed promising results, an F-score of 0.79 for recognizing medical concepts in real-life medical documents.Originality/valueThe originality of the proposed approach lies in the way information is extracted. The context defined through a meta-model proved to be efficient for the task of information extraction, especially from layman descriptions.

TWITTIRÒ: an Italian Twitter Corpus with a Multi-layered Annotation for Irony

Provided the difficulties that still affect a correct identification of irony within the context of Sentiment Analysis tasks, in this paper we describe the main issues emerged during the development of a novel resource for Italian annotated for irony. The project mainly consists in the application on the Twitter corpus TWITTIRÒ of a multi-layered scheme for the fine-grained annotation of irony, as proposed in a multilingual setting and previously applied also on French and English datasets (Karoui et al. 2017). In applying the annotation on this corpus, we outline and discuss the issues and peculiarities emerged about the exploitation of the semantic scheme for Twitter textual messages in Italian, thus shedding some lights on the future directions that can be followed in the multilingual and cross-language perspective too. We present, in particular, an analysis of the annotation process and distribution of the labels of each layer involved in the scheme. This is supported by a discussion of the outcome of the annotation carried on by native Italian speakers in the development of the corpus. In particular, an in-depth discussion of the inter-annotator agreement and of the sources of disagreement is included. The result is a novel gold standard corpus for irony detection in Italian, which enriches the scenario of multilingual datasets available for this challenging task and is ready to be used as a benchmark in automatic irony detection experiments and evaluation campaigns.

Towards the Construction of a Gold Standard Biomedical Corpus for the Romanian Language

Gold standard corpora (GSCs) are essential for the supervised training and evaluation of systems that perform natural language processing (NLP) tasks. Currently, most of the resources used in biomedical NLP tasks are mainly in English. Little effort has been reported for other languages including Romanian and, thus, access to such language resources is poor. In this paper, we present the construction of the first morphologically and terminologically annotated biomedical corpus of the Romanian language (MoNERo), meant to serve as a gold standard for biomedical part-of-speech (POS) tagging and biomedical named entity recognition (bioNER). It contains 14,012 tokens distributed in three medical subdomains: cardiology, diabetes and endocrinology, extracted from books, journals and blogposts. In order to automatically annotate the corpus with POS tags, we used a Romanian tag set which has 715 labels, while diseases, anatomy, procedures and chemicals and drugs labels were manually annotated for bioNER with a Cohen Kappa coefficient of 92.8% and revealed the occurrence of 1877 medical named entities. The automatic annotation of the corpus has been manually checked. The corpus is publicly available and can be used to facilitate the development of NLP algorithms for the Romanian language.

Using semantic context for multiple concepts detection in still images

Multimedia documents indexing systems performances have been improved significantly in recent years, especially after the involvement of deep learning approaches. However, this progress remains insufficient with the evolution of users' needs that become complex in terms of semantics and the number of words that compose their queries. So, it is important to think about indexing images by a group of concepts simultaneously (multi-concepts) and not just single ones. This would allow systems to better respond to queries composed of several terms. This task is much more difficult than indexing images by single concepts. Multi-concepts detection in images has been little dealt in the state of the art compared to the detection of visual single concepts. On the other hand, the use of context has proved its effectiveness in the field of multimedia semantic indexing. In this work, we propose two approaches that consider the semantic context for multi-concepts detection in still images. We tested and evaluated our proposal on the international standard corpus Pascal VOC for the detection of concepts pairs and triplets of concepts. Our contributions have shown that context is useful and improves multi-concepts detection in images. The combination of the use of semantic context and deep learning-based features yielded much better results than those of the state of the art. This difference in performance is estimated by a relative gain on mean average precision reaching + 70% for concepts pairs and + 34% for the case of triplets of concepts.

Ensemble Decision for Spam Detection Using Term Space Partition Approach.

This paper proposes an ensemble decision approach which combines global and local features of e-mails together to detect spam effectively. In the proposed method, a special feature construction method named term space partition (TSP) is utilized to divide the whole term space into several subspaces and adopt different feature construction strategies on each of them, respectively. This method can make each term play a distinct and important role when conducting detection. This method is utilized and extended by introducing the sliding window technique to extract local features from e-mails. The global classifier and local classifiers are constructed on a global feature vector set and local feature vector sets, respectively, and together make the ensemble decision by adopting the voting technique. The principles of the TSP-based approach and mechanism of the ensemble decision method are presented in detail. Five different and standard benchmark corpora are applied to experiments for performance evaluation of this proposed method. Comprehensive experimental results show that the proposed method brings significant performance improvement and better robustness on the basis of the TSP-based approach. In addition, the proposed method outperforms the current prevalent and state-of-the-art approaches, especially when a comprehensive consideration of performance, efficiency, and robustness is taken. This endows it with flexible capability and adaptivity in the real-world applications.

Popularising the Higgs boson: a corpus-assisted approach to reporting scientific discovery in online media

This study explores the scientific popularisation process and how science knowledge is recontextualised and rewritten in the transfer from one context or genre to another genre. It does this through a case study of the discovery of the Higgs boson, a new physics particle that is commonly known as the God Particle, and by focussing on the meta-discursive strategies that emerged from the texts after corpus-assisted analysis. Extensive use was made of exemplification and generalisation through analogies and metaphors, and through ideational content representing epistemic uncertainty in the newspaper discourse. Prominence is given to science popularisation in the British press, because online newspapers capture a wide non-expert public, the aim being to offer an analysis of how the discursive perspective of complex science news (particle physics) is conveyed to the general public, and can allow a systematic investigation into ‘how’ a scientific event is constructed and made newsworthy. Two corpora, consisting of texts from the scientific journal, Physics Letters B, and from online media blogs, were also compiled for contrastive purposes. In this way, prominent lexico-semantic textual properties are identified in the main corpus (containing newspapers) through standard corpus linguistic techniques, in particular through key semantic domain annotation, leading to more insight into how complex science is linguistically constructed and conveyed to a lay audience.

A new hybrid method for Arabic multi-font text segmentation, and a reference corpus construction

In analytical systems of Arabic text recognition, segmentation is a critical and decisive stage. In this stage, words after being extracted from the text document are segmented into individual characters to allow the feature extraction and classification. The absence of standard corpus for printed Arabic text does not allow a good and objective comparison between different segmentation systems. In this paper, we propose two contributions. The first one is a multi-font reference corpus of printed Arabic text, in which we grouped all segmentation problems and which can be used as a reference to compare different segmentation systems. The second is a hybrid method for segmentation of printed multi-font Arabic text. The proposed method is based on two of the most known techniques in the field: contour based and Template matching techniques. This method is insensitive to the font variability and the characters overlapping. To evaluate the results of this method, we have studied, implemented and tested other methods in literature by using the proposed corpus. The experimental results show that our method gives better segmentation rates.

Shared-hidden-layer Deep Neural Network for Under-resourced Language the Content

Training speech recognizer with under-resourced language data still proves difficult. Indonesian language is considered under-resourced because the lack of a standard speech corpus, text corpus, and dictionary. In this research, the efficacy of augmenting limited Indonesian speech training data with highly-resourced-language training data, such as English, to train Indonesian speech recognizer was analyzed. The training was performed in form of shared-hidden-layer deep-neural-network (SHL-DNN) training. An SHL-DNN has language-independent hidden layers and can be pre-trained and trained using multilingual training data without any difference with a monolingual deep neural network. The SHL-DNN using Indonesian and English speech training data proved effective for decreasing word error rate (WER) in decoding Indonesian dictated-speech by achieving 3.82% absolute decrease compared to a monolingual Indonesian hidden Markov model using Gaussian mixture model emission (GMM-HMM). The case was confirmed when the SHL-DNN was also employed to decode Indonesian spontaneous-speech by achieving 4.19% absolute WER decrease.

A standardized corpus for SQLite database forensics

An increasing number of programs like browsers or smartphone apps are using SQLite3 databases to store application data. In many cases, such data is of high value during a forensic investigation. Therefore, various tools have been developed that claim to support rigorous forensic analysis of SQLite database files, claims that are not supported by appropriate evidence. We present a standardized corpus of SQLite files that can be used to evaluate and benchmark analysis methods and tools. The corpus contains databases which use special features of the SQLite file format or contain potential pitfalls to detect errors in forensic programs. We apply our corpus to a set of six available tools and evaluate their strengths and weaknesses. In particular, we show that none of these tools can reliably handle all corner cases of the SQLite3 format.

Toward Binding Database Interfaces with Scientific Papers

Despite a large variety of facilities helping to either select or manipulate data from Web interfaces, it remains diffcult to provide users with relevant scientific or technical annotations for those data. Introducing such content by hand into a Web interface is a tedious job with a risk of providing in complete or inadequate content. To overcome this diffculty, we are exploring the possibility of using the names of exposed quantities to index a text corpus. This index can be used to show the most relevant text snippets in a given context. The full text can be displayed by user request and automatically scroll down to that snippet. Our approach is based on the conversion of PDF papers into machinereadable files that are indexed by a search engine. Index entries are reported as PDF annotations that are used to control the display. This workflow has been tested on the IVOA standard corpus as a proof of concept. It has then been applied to the XMM-Newtonuser guides for our catalog interface. Finally, it has been adapted to find resources within portals exposing a lot of various data collections.

Annotation of phenotypes using ontologies: a gold standard for the training and evaluation of natural language processing systems.

Natural language descriptions of organismal phenotypes, a principal object of study in biology, are abundant in the biological literature. Expressing these phenotypes as logical statements using ontologies would enable large-scale analysis on phenotypic information from diverse systems. However, considerable human effort is required to make these phenotype descriptions amenable to machine reasoning. Natural language processing tools have been developed to facilitate this task, and the training and evaluation of these tools depend on the availability of high quality, manually annotated gold standard data sets. We describe the development of an expert-curated gold standard data set of annotated phenotypes for evolutionary biology. The gold standard was developed for the curation of complex comparative phenotypes for the Phenoscape project. It was created by consensus among three curators and consists of entity–quality expressions of varying complexity. We use the gold standard to evaluate annotations created by human curators and those generated by the Semantic CharaParser tool. Using four annotation accuracy metrics that can account for any level of relationship between terms from two phenotype annotations, we found that machine–human consistency, or similarity, was significantly lower than inter-curator (human–human) consistency. Surprisingly, allowing curatorsaccess to external information did not significantly increase the similarity of their annotations to the gold standard or have a significant effect on inter-curator consistency. We found that the similarity of machine annotations to the gold standard increased after new relevant ontology terms had been added. Evaluation by the original authors of the character descriptions indicated that the gold standard annotations came closer to representing their intended meaning than did either the curator or machine annotations. These findings point toward ways to better design software to augment human curators and the use of the gold standard corpus will allow training and assessment of new tools to improve phenotype annotation accuracy at scale.

Annotating Speaker Stance in Discourse: The Brexit Blog Corpus

AbstractThe aim of this study is to explore the possibility of identifying speaker stance in discourse, provide an analytical resource for it and an evaluation of the level of agreement across speakers. We also explore to what extent language users agree about what kind of stances are expressed in natural language use or whether their interpretations diverge. In order to perform this task, a comprehensive cognitive-functional framework of ten stance categories was developed based on previous work on speaker stance in the literature. A corpus of opinionated texts was compiled, the Brexit Blog Corpus (BBC). An analytical protocol and interface (Active Learning and Visual Analytics) for the annotations was set up and the data were independently annotated by two annotators. The annotation procedure, the annotation agreements and the co-occurrence of more than one stance in the utterances are described and discussed. The careful, analytical annotation process has returned satisfactory inter- and intra-annotation agreement scores, resulting in a gold standard corpus, the final version of the BBC.

A new synonym-substitution method to enrich the human phenotype ontology

BackgroundNamed entity recognition is critical for biomedical text mining, where it is not unusual to find entities labeled by a wide range of different terms. Nowadays, ontologies are one of the crucial enabling technologies in bioinformatics, providing resources for improved natural language processing tasks. However, biomedical ontology-based named entity recognition continues to be a major research problem.ResultsThis paper presents an automated synonym-substitution method to enrich the Human Phenotype Ontology (HPO) with new synonyms. The approach is mainly based on both the lexical properties of the terms and the hierarchical structure of the ontology. By scanning the lexical difference between a term and its descendant terms, the method can learn new names and modifiers in order to generate synonyms for the descendant terms. By searching for the exact phrases in MEDLINE, the method can automatically rule out illogical candidate synonyms. In total, 745 new terms were identified. These terms were indirectly evaluated through the concept annotations on a gold standard corpus and also by document retrieval on a collection of abstracts on hereditary diseases. A moderate improvement in the F-measure performance on the gold standard corpus was observed. Additionally, 6% more abstracts on hereditary diseases were retrieved, and this percentage was 33% higher if only the highly informative concepts were considered.ConclusionsA synonym-substitution procedure that leverages the HPO hierarchical structure works well for a reliable and automatic extension of the terminology. The results show that the generated synonyms have a positive impact on concept recognition, mainly those synonyms corresponding to highly informative HPO terms.

Analysis on Applicability of Common Chinese Word Segmentation Software in Literature Study of Traditional Chinese Medicine Text

This study was aimed to evaluate the applicability of common Chinese word segmentation software used in the literature study of traditional Chinese medicine (TCM) text, in order to put forward ideas on developing specialized TCM text word segmentation software. By means of installing and operating Chinese word segmentation software, the word segmentation experiment was conducted on TCM text samples. Aspects, such as Chinese word segmentation accuracy, speed, maneuverability, reliability, extendibility, portability and other characteristics, were compared among different Chinese word segmentation software. The results showed that there were differences on the accuracy, speed, maneuverability, reliability, extendibility, portability among different Chinese word segmentation software. It was difficult to achieve best performance on different aspects by single software. Through the comparison of different Chinese word segmentation software, the Pan-Gu Segment software showed the best performance on accuracy, with good maneuverability, and high word segmentation efficiency, which was the most suitable for word segmentation in TCM text. It was concluded that developing specialized TCM word segmentation software may be the best solution to meet the requirement of word segmentation in TCM literature study. Basic studies should be strengthened from aspects, such as the construction of standard TCM corpus, the completion of TCM dictionary base, the introduction, optimization and innovation of word segmentation algorithm, as well as the development of word segmentation software for TCM text.

The impact of phonological rules on Arabic speech recognition

The pronunciation variation is a well-known phenomenon that has been widely investigated for automatic speech recognition (ASR). The knowledge-based phonological rules are generally used to capture the accurate phonetic realization in order to minimize the mismatch between the ASR dictionary and the actual phonetic representation of the speech signal. For the Arabic ASR, there are a number of studies that employ these rules on Arabic ASR systems; however, little research has been devoted to measure the precise performance of each rule. In this paper, we aim at finding the exact effect of each rule as well as the rules that have no influence. We used the Carnegie Mellon University PocketSphinx speech recognizer with a new “in-house” modern standard Arabic speech corpus that contains 19 h for training and 3.7 h for testing. We evaluated the effect of three famous rules (Shadda, Tanween, and the solar letters). The experimental results do not show clear evidence that using phonological rules for ASR dictionary adaptation can enhance the performance for within-word pronunciation variation. The obtained results might be an indication to rethink or use other ASR performance aspects, such as cross-word pronunciation variation and the optimal phonemes set of the Arabic language.