Development Of Natural Language Processing Research Articles

Novel Linguistic Steganography Based on Character-Level Text Generation

With the development of natural language processing, linguistic steganography has become a research hotspot in the field of information security. However, most existing linguistic steganographic methods may suffer from the low embedding capacity problem. Therefore, this paper proposes a character-level linguistic steganographic method (CLLS) to embed the secret information into characters instead of words by employing a long short-term memory (LSTM) based language model. First, the proposed method utilizes the LSTM model and large-scale corpus to construct and train a character-level text generation model. Through training, the best evaluated model is obtained as the prediction model of generating stego text. Then, we use the secret information as the control information to select the right character from predictions of the trained character-level text generation model. Thus, the secret information is hidden in the generated text as the predicted characters having different prediction probability values can be encoded into different secret bit values. For the same secret information, the generated stego texts vary with the starting strings of the text generation model, so we design a selection strategy to find the highest quality stego text from a number of candidate stego texts as the final stego text by changing the starting strings. The experimental results demonstrate that compared with other similar methods, the proposed method has the fastest running speed and highest embedding capacity. Moreover, extensive experiments are conducted to verify the effect of the number of candidate stego texts on the quality of the final stego text. The experimental results show that the quality of the final stego text increases with the number of candidate stego texts increasing, but the growth rate of the quality will slow down.

The Translator’s Extended Mind

The rapid development of natural language processing in the last three decades has drastically changed the way professional translators do their work. Nowadays most of them use computer-assisted translation (CAT) or translation memory (TM) tools whose evolution has been overshadowed by the much more sensational development of machine translation (MT) systems, with which TM tools are sometimes confused. These two language technologies now interact in mutually enhancing ways, and their increasing role in human translation has become a subject of behavioral studies. Philosophers and linguists, however, have been slow in coming to grips with these important developments. The present paper seeks to fill in this lacuna. I focus on the semantic aspects of the highly distributed human–computer interaction in the CAT process which presents an interesting case of an extended cognitive system involving a human translator, a TM tool, an MT engine, and sometimes other human translators or editors. Considered as a whole, such a system is engaged in representing the linguistic meaning of the source document in the target language. But the roles played by its various components, natural as well as artificial, are far from trivial, and the division of linguistic labor between them throws new light on the familiar notions that were initially inspired by rather different phenomena in the philosophy of language, mind, and cognitive science.

Relating Mori's Uncanny Valley in generating conversations with artificial affective communication and natural language processing.

Human beings express affinity (Shinwa-kan in Japanese language) in communicating transactive engagements among healthcare providers, patients and healthcare robots. The appearance of healthcare robots and their language capabilities often feature characteristic and appropriate compassionate dialogical functions in human-robot interactions. Elements of healthcare robot configurations comprising its physiognomy and communication properties are founded on the positivist philosophical perspective of being the summation of composite parts, thereby mimicking human persons. This article reviews Mori's theory of the Uncanny Valley and its consequent debates, and examines "Uncanny" relations with generating healthcare robot conversational content with artificial affective communication (AAC) using natural language processing. With healthcare robots provoking influential physical composition and sensory expressions, the relations in human-healthcare robot transactive engagements are argued as supportive of the design and development in natural language processing. This implies that maintaining human-healthcare robot interaction and assessing the eeriness situations explained in the Uncanny Valley theory are crucial positions for healthcare robot functioning as a valuable commodity in health care. As such, physical features, language capabilities and mobility of healthcare robots establish the primacy of the AAC with natural language processing as integral to healthcare robot-human healthcare practice.

Generation and evaluation of artificial mental health records for Natural Language Processing

A serious obstacle to the development of Natural Language Processing (NLP) methods in the clinical domain is the accessibility of textual data. The mental health domain is particularly challenging, partly because clinical documentation relies heavily on free text that is difficult to de-identify completely. This problem could be tackled by using artificial medical data. In this work, we present an approach to generate artificial clinical documents. We apply this approach to discharge summaries from a large mental healthcare provider and discharge summaries from an intensive care unit. We perform an extensive intrinsic evaluation where we (1) apply several measures of text preservation; (2) measure how much the model memorises training data; and (3) estimate clinical validity of the generated text based on a human evaluation task. Furthermore, we perform an extrinsic evaluation by studying the impact of using artificial text in a downstream NLP text classification task. We found that using this artificial data as training data can lead to classification results that are comparable to the original results. Additionally, using only a small amount of information from the original data to condition the generation of the artificial data is successful, which holds promise for reducing the risk of these artificial data retaining rare information from the original data. This is an important finding for our long-term goal of being able to generate artificial clinical data that can be released to the wider research community and accelerate advances in developing computational methods that use healthcare data.

A method of computing conceptual semantic similarity based on part-whole relationship

Conceptual semantic similarity calculation is an important content in research fields such as natural language processing, machine translation and is widely used in information retrieval, knowledge acquisition, text classification, and clustering. It’s very important to development of natural language processing by improving the accuracy of semantic similarity calculation. This paper proposes a method for calculating the similarity of concepts based on the part-whole relationship between concepts. This method is based on the partial overall relationship diagram and uses the hierarchical features and positions of the concepts in the diagram to calculate the conceptual similarity.

TS-CSW: text steganalysis and hidden capacity estimation based on convolutional sliding windows

With the rapid development of natural language processing (NLP) technology in the past few years, the automatic steganographic texts generation methods have been greatly developed. Benefiting from the powerful feature extraction and expression capabilities of neural networks, these methods can generate steganographic texts with both relatively high concealment and high hidden capacity at the same time. For these steganographic methods, previous steganalysis models show unsatisfactory detection performance, which remains an unsolved problem and poses a great threat to the security of cyberspace. In this paper, we first collect a large text steganalysis (T-Steg) dataset, which contains a total number of 396,000 texts with various embedding rates under various formats. We analyze that there are three kinds of word correlation patterns in texts. Then we propose a new text steganalysis model based on convolutional sliding windows (TS-CSW), which use convolutional sliding windows (CSW) with multiple sizes to extract those correlation features. We observed that these word correlation features in the generated steganographic texts would be distorted after being embedded with secret information. These subtle changes of correlation feature distribution could then be used for text steganalysis. We use the samples collected in T-Steg dataset to train and test the proposed steganalysis method. Experimental results show that the proposed model can not only achieve a high steganalysis performance, but can even estimate the amount of secret information embedded in the generated steganographic texts, which shows a state-of-the-art performance.

An Intelligent Chatbot System Based on Entity Extraction Using RASA NLU and Neural Network

Intelligent chatbot systems are popular issues in the application fields of robot system and natural language processing. As the development of natural language processing and neural network algorithms, the application of artificial intelligence is increasing in Chatbot systems, which are typically used in dialog systems for various practical purposes including customer service or information acquisition. This paper designs the functional framework and introduces the principle of RASA NLU for the Chatbot system, then it integrates RASA NLU and neural network (NN) methods and implements the system based on entity extraction after intent recognition. With the experimental comparison and validation, our developed system can realize automatic learning and answering the collected questions about finance. The system analysis of two methods also validate that RASA NLU outperforms NN in accuracy for a single experiment, but NN has better integrity to classify entities from segmented words.

A Natural Language Processing Pipeline of Chinese Free-Text Radiology Reports for Liver Cancer Diagnosis

Despite the rapid development of natural language processing (NLP) implementation in electronic medical records (EMRs), Chinese EMRs processing remains challenging due to the limited corpus and specific grammatical characteristics, especially for radiology reports. In this study, we designed an NLP pipeline for the direct extraction of clinically relevant features from Chinese radiology reports, which is the first key step in computer-aided radiologic diagnosis. The pipeline was comprised of named entity recognition, synonyms normalization, and relationship extraction to finally derive the radiological features composed of one or more terms. In named entity recognition, we incorporated lexicon into deep learning model bidirectional long short-term memory-conditional random field (BiLSTM-CRF), and the model finally achieved an F1 score of 93.00%. With the extracted radiological features, least absolute shrinkage and selection operator and machine learning methods (support vector machine, random forest, decision tree, and logistic regression) were used to build the classifiers for liver cancer prediction. For liver cancer diagnosis, random forest had the highest predictive performance in liver cancer diagnosis (F1 score 86.97%, precision 87.71%, and recall 86.25%). This work was a comprehensive NLP study focusing on Chinese radiology reports and the application of NLP in cancer risk prediction. The proposed NLP pipeline for the radiological feature extraction could be easily implemented in other kinds of Chinese clinical texts and other disease predictive tasks.

Research on Mongolian-Chinese machine translation based on the end-to-end neural network

With the development of natural language processing and neural machine translation, the neural machine translation method of end-to-end (E2E) neural network model has gradually become the focus of research because of its high translation accuracy and strong semantics of translation. However, there are still problems such as limited vocabulary and low translation loyalty, etc. In this paper, the discriminant method and the Conditional Random Field (CRF) model were used to segment and label the stem and affixes of Mongolian in the preprocessing stage of Mongolian-Chinese bilingual corpus. Aiming at the low translation loyalty problem, a decoding model combining Convolution Neural Network (CNN) and Gated Recurrent Unit (GRU) was constructed. The target language decoding was performed by using the GRU. A global attention model was used to obtain the bilingual word alignment information in the process of bilingual word alignment processing. Finally, the quality of the translation was evaluated by Bilingual Evaluation Understudy (BLEU) values and Perplexity (PPL) values. The improved model yields a BLEU value of 25.13 and a PPL value of [Formula: see text]. The experimental results show that the E2E Mongolian-Chinese neural machine translation model was improved in terms of translation quality and semantic confusion compared with traditional statistical methods and machine translation models based on Recurrent Neural Networks (RNN).

Language theoretic properties of regular DAG languages

We study sets of directed acyclic graphs, called regular DAG languages, which are accepted by a recently introduced type of DAG automata motivated by current developments in natural language processing. We prove (or disprove) closure properties, establish pumping lemmata, characterize finite regular DAG languages, and show that “unfolding” turns regular DAG languages into regular tree languages, which implies a linear growth property and the regularity of the path languages of regular DAG languages. Further, we give polynomial decision algorithms for the emptiness and finiteness problems, and show that deterministic DAG automata can be minimized and tested for equivalence in polynomial time.

Contex Free Grammer For Turkish

Formal Grammar which is introduced by Chomsky is one of the most important development in Natural Language Processing, a branch of Artificial Intelligence. The mathematical reresentation of languages can be possible using Formal Grammars. Almost all natural languages have word classes such as noun, adjective, verb. In addition to this one sentence consist of noun phrase and verb phrase. Noun phrase may consist of location, destination and source elements. Despite many similarities between the languages, there exist important dissimilarities in grammar rules of the languages belonging to different language families. In our study the most appropriate formal grammar representing Turkish language is investigated. Accuracy of the suggested grammars’ rules is evaluated in two different corpus. This study is the enhanced version of “Turkish Context Free Grammar Rules with Case Suffix and Phrase Relation” that was presented on UBMK 2016 International Conference on Computer Science \& Engineering \cite{ilk}. Different from the first study, this study includes all word and sentence types of Turkish. Adjectives and prepositions are considered. The quoted sentences, incomplete sentences and question sentences are included. The genitive phrase structures including verbal word are included. In this study, the noun phrases are also defined in detail.

Automated Sentence Boundary Detection in Modern Standard Arabic Transcripts using Deep Neural Networks

The increased volumes of Arabic sources of data available on the Web has boosted the development of Natural Language Processing (NLP) tools over different tasks and applications. However, to take advantage from a vast amount of these applications, a prior segmentation task call Sentence Boundary Detection (SBD) is needed. In this paper we focus on SBD over Modern Standard Arabic (MSA) by comparing two different approaches based on Deep Neural Networks (DNN) using out-of-domain and in-domain training data with only lexical features (represented as character embedding) while conducting two scenarios based on a Convolutional Neural Network and a Recurrent Neural Network with attention mechanism architectures. While tuning a big out-of-domain dataset with a smaller in-domain dataset, improves the performance in general. Our evaluations were based on IWSLT 2017 TED talks transcripts and showed similarities and differences depending of the SBD method. MSA carries certain complications given its rich and complex morphology. However, using only lexical features for Arabic SBD is an acceptable option when the source audio signal is not available and a certain level of language independence needs to be reached.

Normalizing acronyms and abbreviations to aid patient understanding of clinical texts: ShARe/CLEF eHealth Challenge 2013, Task 2.

BackgroundThe ShARe/CLEF eHealth challenge lab aims to stimulate development of natural language processing and information retrieval technologies to aid patients in understanding their clinical reports. In clinical text, acronyms and abbreviations, also referenced as short forms, can be difficult for patients to understand. For one of three shared tasks in 2013 (Task 2), we generated a reference standard of clinical short forms normalized to the Unified Medical Language System. This reference standard can be used to improve patient understanding by linking to web sources with lay descriptions of annotated short forms or by substituting short forms with a more simplified, lay term.MethodsIn this study, we evaluate 1) accuracy of participating systems’ normalizing short forms compared to a majority sense baseline approach, 2) performance of participants’ systems for short forms with variable majority sense distributions, and 3) report the accuracy of participating systems’ normalizing shared normalized concepts between the test set and the Consumer Health Vocabulary, a vocabulary of lay medical terms.ResultsThe best systems submitted by the five participating teams performed with accuracies ranging from 43 to 72 %. A majority sense baseline approach achieved the second best performance. The performance of participating systems for normalizing short forms with two or more senses with low ambiguity (majority sense greater than 80 %) ranged from 52 to 78 % accuracy, with two or more senses with moderate ambiguity (majority sense between 50 and 80 %) ranged from 23 to 57 % accuracy, and with two or more senses with high ambiguity (majority sense less than 50 %) ranged from 2 to 45 % accuracy. With respect to the ShARe test set, 69 % of short form annotations contained common concept unique identifiers with the Consumer Health Vocabulary. For these 2594 possible annotations, the performance of participating systems ranged from 50 to 75 % accuracy.ConclusionShort form normalization continues to be a challenging problem. Short form normalization systems perform with moderate to reasonable accuracies. The Consumer Health Vocabulary could enrich its knowledge base with missed concept unique identifiers from the ShARe test set to further support patient understanding of unfamiliar medical terms.

Facilitate Knowledge Exploration with Storytelling

With the fast development of natural language processing and web technologies in recent years, the amount of knowledge/data available for everyone to explore has grown exponentially. This project is aimed at creating an interactive and proactive presentation agent. In particular, narrative and storytelling techniques are used for engaging the audience and helping the audience digest and remember the content. We present examples of using this system for generating presentations, and preliminary evaluation results.

An Extension of Standard Latent Dirichlet Allocation to Multiple Corpora

Latent Dirichlet Allocation (LDA) is a highly successful topic modeling framework. We describe a new extension to LDA which supports multiple subcorpora, each containing a different type of document. As in LDA, this multiple-corpora LDA (mLDA) model assumes document topic proportions follow a symmetric Dirichlet distribution. However, in mLDA, the Dirichlet parameter is subcorpus dependent. An online algorithm for training mLDA models is derived. The algorithm is applied to data from the USC Shoah Foundation’s Visual History Archive. Results show mLDA produced a better language model than standard LDA for this data. Using the same data, the mLDA topic model is used to construct an information retrieval system. Search results from this system outperform those obtained from traditional string-based search systems. A novel approach to the visualization of topics is outlined and visualizations are presented. As a novel development in natural language processing, mLDA will allow the power of topic modeling to be applied to a huge range of fields with diverse data by incorporating more information into a single topic model. It also enhances the applicability of topic modeling to information retrieval.

Tools to accurately identify veterans who undergo molecular diagnostic testing.

201 Background: Yearly, 52,000 veterans are diagnosed with cancer. Minorities represent 18.5% (9,651) of these patients. The rapid and explosive growth of molecular diagnostics (MDx) has created challenges for large healthcare systems such as the Veterans Health Administration (VA) to integrate genomic data into the EMR. Yet, this is an important component of high quality cancer care. In 2011, the VA cancer registry (VACCR) began reporting molecular data. This presentation will describe one project to improve integration of genomics into the EMR and the VACCR. Methods: Using ICD diagnosis codes, we identified veterans diagnosed in 2011 with brain, breast, colon, gastrointestinal stromal, lung, and melanoma cancers. Administrative data was obtained to identify MDx testing. These data were then compared to random chart audits. Significant discrepancies between these sources of data prompted collaboration with national and proprietary reference labs (ARUP, LabCorp, Quest, Genomic Health) to obtain the volume of testing by each VAMC. These data were used to conduct targeted chart reviews to identify the location, processes of care, free and structured text information. These data informed the development of natural language processing (NLP) tools to automatically identify patients that underwent testing. Results: Laboratories had the most accurate source of data. Data from ARUP, Quest, and LabCorp identified a significantly higher volume of testing than reported by administrative data. Applying NLP tools to patients diagnosed with breast cancer identified 44 of the 116 tests ordered for the 21-gene risk score tests. Conclusions: Decision support systems are needed to link tumor SNOMED code to diagnostic testing. Until systems are developed, collaborations with reference labs may be an effective method for identifying molecular data. NLP tools may also serve as an adjunct method for capturing MDx tests ordered from smaller labs.

A survey on the exchange of linguistic resources

PurposeThe purpose of this paper is to provide a literature review of the principal formats and frameworks that have been used in the last 20 years to exchange linguistic resources. It aims to give special attention to the most recent approaches to publishing linguistic linked open data on the Web.Design/methodology/approachResearch papers published since 1990 on the use of various formats, standards, frameworks and methods to exchange linguistic information were divided into two main categories: those proposing specific schemas and syntaxes to suit the requirements of a given type of linguistic data (these are referred to as offline approaches), and those adopting the linked data (LD) initiative and the semantic web technologies to support the interoperability of heterogeneous linguistic resources. For each paper, the type of linguistic resource exchanged, the framework/format used, the interoperability approach taken and the related projects were identified.FindingsThe information gathered in the survey reflects an increase in recent years in approaches adopting the LD initiative. This is due to the fact that the structural and syntactic issues which arise when addressing the interoperability of linguistic resources can be solved by applying semantic web technologies. What remains an open issue in the field of computational linguistics is the development of knowledge artefacts and mechanisms to support the alignment of the different aspects of linguistic resources in order to guarantee semantic and conceptual interoperability in the linked open data (LOD) cloud. Ontologies have proved to be of great use in achieving this goal.Research limitations/implicationsThe research presented here is by no means a comprehensive or all‐inclusive survey of all existing approaches to the exchange of linguistic resources. Rather, the aim was to highlight, analyze and categorize the most significant advances in the field.Practical implicationsThis survey has practical implications for computational linguists and for every application requiring new developments in natural language processing. In addition, multilingual issues can be better addressed when semantic interoperability of heterogeneous linguistic resources is achieved.Originality/valueThe paper provides a survey of past and present research and developments addressing the interoperability of linguistic resources, including those where the linked data initiative has been adopted.

NLP@Desktop

Research and development in Natural Language Processing (NLP) has made significant progress over the last decade. Many robust NLP systems have been developed for handling machine translation, question-answering, summarization, topic detection, cluster analysis, information extraction, named entity recognition (NER), etc. Despite this advancement in NLP research, the results are still not accessible for common desktop users. In the current scenario, it is difficult to integrate a new NLP tool with the existing text processing applications. To overcome this, we have implemented a service deployer framework based on DBus[1] for the seamless integration of NLP applications with desktop clients like email clients, word processors, browsers etc. This framework enables NLP researchers to make their products equipped for the desktop clients with minimal efforts.

Language and the Boundaries of Research: Media Monitoring Technologies in International Media Research

This article argues that data-driven research in global media has not kept up with the inherent interest in internationalization due to a number of factors, including access to global media and the lack of foreign language skills among many media researchers. Thus, the field still remains focused on analysis of Western media, leading to misunderstanding the geopolitical impact of global media. The article explores emerging media monitoring technologies which have arisen out of developments in natural language processing and machine generated translation. Finally, the article explores innovative research programs that utilize these technologies for global media research and education.

Use of NLP Tools in CALL System for Arabic

This article focuses on the development of Natural Language Processing (NLP) tools for Computer Assisted Language Learning (CALL). First, we have developed some NLP tools: a labelled dictionary of Arabic (as complete as possible), a generator for morphological derivatives, a Conjugator and a morphological analyzer for Arabic. Second, we used these tools to create a number of educational applications for learning the Arabic language by using the proposed system SALA (an NLP-based authoring system, organized into three distinct layers: functions, scripts and activities).