Form Of Natural Language Text Research Articles

Named Entity Recognition in Russian Using Multi-Task LSTM-CRF

Named entity recognition (NER) is aimed at obtaining the important information from the unstructured data presented in the form of natural language texts. In this work, we investigate the efficiency of modern multi-task NER approaches on Russian language corpora by employing several different NER datasets and a dataset of part-of-speech (POS) tags. We apply a state-of-the-art neural architecture based on bidirectional LSTMs and conditional random fields. Convolutional neural networks were utilized to learn character-level features. We carry out extensive experimental evaluation over three standard datasets of news articles written in Russian. The proposed multi-task model achieve state-of-the-art results with an F1 score of 88.04% on Gareev’s dataset and an F1 score of 99.49% on Person-1000 dataset.

Radio map construction based on BERT for fingerprint-based indoor positioning system

Due to the heavy workload of RSS collection, the instability of WLAN signal strength and the disappearance of signals caused by complex indoor environments, the construction of radio map for wireless local area network (WLAN) fingerprint-based indoor positioning system is time-consuming and laborious. In order to rapidly deploy indoor WLAN positioning system, the bidirectional encoder representation from transformers (BERT) model is used to fill the missing signal in radio map and quickly build radio map. The radio map is imported into the BERT model in the form of natural language text, and the missing signal is filled by the BERT model. Since the number of input data in BERT model cannot exceed 512 words, the structure of BERT model is not suitable for WLAN signals with large data volume. Therefore, we redefine the model structure based on the original BERT model and fill in the missing signals in the radio map in parallel. In addition, the loss function is redefined. Except that each segment has a loss function, the weighted average value of all segment loss functions is defined as the total loss function. The experimental results show that the BERT model is better than the traditional linear interpolation method, compressed sensing algorithm and matrix completion algorithm in filling the missing signals in the fingerprint database, and the probability of error within 2 m reaches about 94%.

Tactics And Techniques Classification In Cyber Threat Intelligence

Abstract Completing the classification of tactics and techniques in cyber threat intelligence (CTI) is an important way to obtain tactics, techniques and procedures (TTPs) and portray the behavior of cyber attacks. However, the high level of abstraction of tactics and techniques information and their presence in CTI, usually in the form of natural language text, make it difficult for traditional manual analysis methods and feature engineering-based machine learning methods to complete the classification of tactics and techniques effectively. Meanwhile, flat deep learning methods do not perform well in classifying more fine-grained techniques due to their inability to exploit the hierarchical relationship between tactics and techniques. Therefore, this paper regards the tactics and techniques of TTPs defined in Adversarial Tactics, Techniques and Common Knowledge knowledge base as labels and proposes a Convolutional Neural Network (CNN) model based on hierarchical knowledge migration and attention mechanism for classifying tactics and techniques in CTI, named HM-ACNN (CNN based on hierarchical knowledge migration and attention mechanism). HM-ACNN classifies tactics and techniques into two phases, and the underlying network model for both phases is the Attention-based CNN network. The first step in HM-ACNN is converting the CTI text into a two-dimensional image based on the word embedding model, and then start training the classification of tactics through the CNN structure based on the attention mechanism before the classification of techniques. Secondly, after the tactics classification training is completed, the tactic-to-technique knowledge migration is then completed by transforming the parameters of the CNN layer and the attention layer in the tactics classification process based on the special hierarchical relationship between tactics and techniques. Then, the classification of techniques is finished by fine-tuning. The experimental results show that HM-ACNN performs well in the tactics and techniques classification tasks, and the metric F1 values reach 93.66% and 86.29%, which are better than other models such as CNN, Recurrent Neural Network and CRNN (Recurrent Convolutional Neural Networks).

RECURSIVE REDUCTION METHOD AS A COMPONENT OF LINGUISTIC SUPPORT OF INFORMATION-ANALYTICAL SYSTEMS

Our time is characterized by a rapid increase in the amount of available information, which leads to the creation of large-scale arrays of thematically and spatially distributed information. Manual processing of such arrays is an extremely complex process that requires significant labor costs. Such arrays can be processed with the help of modern information-analytical systems, in particular – lexicographic ones. However, a significant part of the information in such arrays may be weakly structured or unstructured (in particular, in the form of natural language texts), which creates a need for high-quality linguistic support for such systems as an element of applied linguistics. There is a large number of methods and tools for processing natural language texts – based on dictionaries, machine learning, statistical indicators, etc. However, the results of all these methods require further processing and conversion into the format required by the system in use. For such processing the method of recursive reduction is proposed, which can be used as a component of the linguistic support of information-analytical systems. The method involves the creation of a formalized description of the data format required by the system, based on which the results of the analysis of the input natural language text are transformed, with a creation of an ontology. The resulting ontology can be used to form an ontology-driven lexicographic system.

Design of Relation Extraction Framework to Develop Knowledge Base

Background: Web documents display information in the form of natural language text which is not understandable by machines. To search specific information from sea of web documents has become very challenging as it shows many unwanted non relevant documents along with relevant documents. To retrieve relevant information semantic knowledge can be stored in the domain specific ontology which helps in understanding user’s need to retrieve relevant information. Methods: In this paper, framework for extracting and visualising semantic knowledge has been designed. Proposed approach is based on the assumption that semantics of text can be extracted by creating syntactic structure of the text. To extracts syntactic structure Stanford parser has been used. Parsing of corpus text is done to obtain morphological structures which is in more machine readable format, and thus provides a better structure for constructing syntactic-semantic rules manually. The tagged form of each sentence is taken and set of rules based on dependency relationship are build manually. Sentence level analysis is performed for concepts generation, for properties and for hierarchical relation extraction using dependency parse tree as a means for relation extraction. Results: Extracted concepts and relation among various entities constitute knowledge base in the form of ontology. Conclusion: Proposed information extraction model successfully filter the desired information from the large ocean of internet and create semantic structure to represent data in standard machine understandable format which explain the details about entities along with their properties and their relationship.

The case for NLP-enhanced database tuning

A large body of knowledge on database tuning is available in the form of natural language text. We propose to leverage natural language processing (NLP) to make that knowledge accessible to automated tuning tools. We describe multiple avenues to exploit NLP for database tuning, and outline associated challenges and opportunities. As a proof of concept, we describe a simple prototype system that exploits recent NLP advances to mine tuning hints from Web documents. We show that mined tuning hints improve performance of MySQL and Postgres on TPC-H, compared to the default configuration.

Trainable Framework for Information Extraction, Structuring and Summarization of Unstructured Data, Using Modified NER

The World Wide Web is an ever expanding source of data in today’s world. Millions of tera-bytes of data and information is getting added every second. In this information age as the data is getting generated at an exponential rate, the fact to be noted is that most of the information is already available is in the form of natural language text. The task of information extraction from mammoth data leads us to think on the quality and the form of available data. Secondly, the ever increasing data poses a challenging task of extracting useful information from the available data. The third task is to extract information as efficiently as possible. For retrieving the information there is a need to develop ingenious way to answer any kind of query put up by a user from the available unstructured data. This paper proposes a novel trainable and integrated Natural Language Information Interpretation and Representation System (NLIIRS) that accepts any available un-annotated corpus of data in the form of natural language, and performs the following tasks: finds out the useful data, extracts relevant information in usable form (structured form/tables), summarizes the data and structures the data in relational form. At the end the Question and Answering (Q&A) module shows the cognitive abilities of NLIIR system by answering the questions in natural language relevant to the text. This multispecialty system beyond just Q&A. This is a trainable system capable of handling any unstructured data to be transformed into structured and well organized information. It allows the user to ask questions in natural language. It adopts the advantages of a modified named entity recognition so as to bypass the time consuming process of parts of speech tagging while pre-processing the available corpus (data) for information extraction.

Implementation of Effective Models for Classifying Medical Data Using Text Mining

The paper is devoted to the development and implementation of effective models of medical data classification by text mining for decision support in the diagnosis of pulmonological diseases in children and adolescents of the Altai Territory. Medical data contains important information about patients. Test results are usually retained as structured data, but some data are retained in the form of natural language texts (medical history, the results of physical examination, and the results of other examinations). The paper assesses the quality of the developed methods for extracting information from clinical texts. An assessment of the method for the automatic diagnosis of pulmonological diseases in a test sample is conducted. The most informative features, as well as suitable machine learning methods for classifying patients by disease groups, are identified. Many tasks arising in clinical practice can be automated by applying methods for intelligent analysis of structured and unstructured data that will lead to improvement of the healthcare quality. The results of the research indicate the prospect of using models to support decisionmaking in the primary diagnosis of pulmonological diseases in children and adolescents of the Altai Territory.

RETRACTED ARTICLE: An integrated approach towards automated software requirements elicitation from unstructured documents

Enormous amount of text documents that contains requirements expressed in form of unstructured text are obtained in big software development projects and they have to be transformed into structured documents. Collecting requirements from such huge volume of informal text is still a key mission to be achieved. The majority of the software requirements data that are available to software engineers are articulated in form of Natural Language text and there is no formal grammar or specified syntax are followed by the customer for writing requirements i.e. they are highly unstructured. Manual requirements elicitation from these unstructured documents takes lot of time, it is also subject to errors and it is tedious to perform. Any deficiency in the process of gathering the requirements will lead to undesirable consequences that affects the software development. Requirements that are confusing and not complete in any aspect can collapse the software development project itself. We focus on developing an integrated framework towards automated software requirements elicitation from unstructured documents. A formal grammar based technique is developed to tackle the complex sentence structure, regular expression grammar rules that analyzes the linguistic elements in unstructured text are developed and applied to extract the core concepts/entities and relations between entities, the building blocks of any information system development. Another important task that contributes for automation is to build a domain ontology consisting of concepts, their attributes and relation between concepts and axioms, it extends guidance in the semantic labelling of the extracted information constructs. Since it is expensive to have a domain expert to be present all through the stages of software requirements elicitation, domain ontology serves as a background knowledge source. The extracted requirements after semantic association are then presented to the user in form of Object Role Model diagrams to obtain their consent. The framework provides an overall accuracy of 91% in terms of semantic mapping of the extracted requirements.

Named-Entity Recognition Using Automatic Construction of Training Data From Social Media Messaging Apps

In recent years, social media messaging app data has served as a precious resource to extract useful information, such as critical clues and evidence in legal trials and criminal investigations. Although these data can be of various types, they are mostly in the form of natural language text. Therefore, to extract information from them efficiently, it is essential to research practical natural language processing approaches. This study proposes applying a deep-learning-based named-entity recognition (NER) system as a natural language processing approach for information extraction to these messaging data. In addition, a system for automatically constructing NER training data is presented using the distant supervision method for the training data of deep-learning models. Because social media messaging app data generally include a significant amount of noise, such as typographical and word-spacing errors, a NER system with robustness against these types of noisy data is required to extract information from the messaging data effectively. The results demonstrate that the proposed approach outperforms that of a NER system with manually labeled training data.

A Information Retrieval Based on Question and Answering and NER for Unstructured Information Without Using SQL

In today’s world, the availability of information in the form of unstructured data is in abundance. The unstructured information received is more often than not in the form of natural language text. For any defense establishment, the spy data or any sensitive information received may be best utilized when the information can be extracted efficiently and easily. The proposed model is applicable wherever the influx of text-heavy (unstructured data) is high like the information from the world wide web, documents related to a particular domain, or any other source where the information is in the form of natural language. The proposed Natural Language Information Interpretation and Representation System (NLIIRS) accepts the information in the form of natural language text, processes the information and allows the user to retrieve information by rendering questions in natural language. The questions thus asked by the user are responded by NLIIRS in the form of factoid or phrase based answers. In comparison to the conventional question and answering systems the proposed NLIIRS uses the advantages of both named entity recognition as well as sequential pattern matching based answer search technique. The proposed technique helps us to avoid the use of structured query language (SQL) at the back-end for information processing, storage and extraction. The conversion of user query to SQL statements and also storing the unstructured text in the form of relation tables can be avoided by using NLIIRS. By using this approach in our novel text processing algorithm, after every execution step, the pattern matching and extraction process of the answers to the queries becomes concise and faster. The whole system has been designed on natural language tool kit of Stanford University which helped us to generate parts of speech tag, tokenize the data, and forming tree structure. The novel text processing algorithm utilizes the lemmatizer, stemmer and ne_chunker to prepare the text for information retrieval via Q&A. The advantage of this system is that it does not need training. This system will enable the user to retrieve any information of his/her choice from the available unstructured information.

SEthesaurus: WordNet in Software Engineering

Informal discussions on social platforms (e.g., Stack Overflow, CodeProject) have accumulated a large body of programming knowledge in the form of natural language text. Natural language process (NLP) techniques can be utilized to harvest this knowledge base for software engineering tasks. However, consistent vocabulary for a concept is essential to make an effective use of these NLP techniques. Unfortunately, the same concepts are often intentionally or accidentally mentioned in many different morphological forms (such as abbreviations, synonyms and misspellings) in informal discussions. Existing techniques to deal with such morphological forms are either designed for general English or mainly resort to domain-specific lexical rules. A thesaurus, which contains software-specific terms and commonly-used morphological forms, is desirable to perform normalization for software engineering text. However, constructing this thesaurus in a manual way is a challenge task. In this paper, we propose an automatic unsupervised approach to build such a thesaurus. In particular, we first identify software-specific terms by utilizing a software-specific corpus (e.g., Stack Overflow) and a general corpus (e.g., Wikipedia). Then we infer morphological forms of software-specific terms by combining distributed word semantics, domain-specific lexical rules and transformations. Finally, we perform graph analysis on morphological relations. We evaluate the coverage and accuracy of our constructed thesaurus against community-cumulated lists of software-specific terms, abbreviations and synonyms. We also manually examine the correctness of the identified abbreviations and synonyms in our thesaurus. We demonstrate the usefulness of our constructed thesaurus by developing three applications and also verify the generality of our approach in constructing thesauruses from data sources in other domains.

Mining Structures of Factual Knowledge from Text: An Effort-Light Approach

Abstract Real-world data, though massive, is largely unstructured in the form of natural-language text. It is challenging but highly desirable to mine structures from massive text data, without ext...

Technologies for Complex Intelligent Clinical Data Analysis

The paper presents the system for intelligent analysis of clinical information. Authors describe methods implemented in the system for clinical information retrieval, intelligent diagnostics of chronic diseases, patient's features importance and for detection of hidden dependencies between features. Results of the experimental evaluation of these methods are also presented. Healthcare facilities generate a large flow of both structured and unstructured data which contain important information about patients. Test results are usually retained as structured data but some data is retained in the form of natural language texts (medical history, the results of physical examination, and the results of other examinations, such as ultrasound, ECG or X-ray studies). Many tasks arising in clinical practice can be automated applying methods for intelligent analysis of accumulated structured array and unstructured data that leads to improvement of the healthcare quality. the creation of the complex system for intelligent data analysis in the multi-disciplinary pediatric center. Authors propose methods for information extraction from clinical texts in Russian. The methods are carried out on the basis of deep linguistic analysis. They retrieve terms of diseases, symptoms, areas of the body and drugs. The methods can recognize additional attributes such as "negation" (indicates that the disease is absent), "no patient" (indicates that the disease refers to the patient's family member, but not to the patient), "severity of illness", disease course", "body region to which the disease refers". Authors use a set of hand-drawn templates and various techniques based on machine learning to retrieve information using a medical thesaurus. The extracted information is used to solve the problem of automatic diagnosis of chronic diseases. A machine learning method for classification of patients with similar nosology and the methodfor determining the most informative patients'features are also proposed. Authors have processed anonymized health records from the pediatric center to estimate the proposed methods. The results show the applicability of the information extracted from the texts for solving practical problems. The records ofpatients with allergic, glomerular and rheumatic diseases were used for experimental assessment of the method of automatic diagnostic. Authors have also determined the most appropriate machine learning methods for classification of patients for each group of diseases, as well as the most informative disease signs. It has been found that using additional information extracted from clinical texts, together with structured data helps to improve the quality of diagnosis of chronic diseases. Authors have also obtained pattern combinations of signs of diseases. The proposed methods have been implemented in the intelligent data processing system for a multidisciplinary pediatric center. The experimental results show the availability of the system to improve the quality of pediatric healthcare.

Semi-Supervised Classification of Patient Safety Event Reports

The Veterans Health Administration patient safety reporting system receives more than 100,000 reports annually. The information contained in these reports is primarily in the form of natural language text. Improving the ability to efficiently mine these patient safety reports for information is the objective of a proposed semi-supervised method. A semi-supervised classification method leverages information from both labeled and unlabeled reports to predict categories for the unlabeled reports. Two different scenarios involving a semi-supervised learning process are examined, and both demonstrate good predictive results. The semi-supervised method shows much promise in assisting researchers and analysts toward accurately and more quickly separating reports of varying and often overlapping topics. The method is able to use the "stories" provided in patient safety reports to extend existing patient safety taxonomies beyond their static design.

Computer assisted text analysis in the social sciences

We use the term “Computer Assisted Text Analysis” in a broad sense to refer to a range of current techniques from quantitative social science and content analysis to ‘data mining’ and ‘text classification’, including the analysis of open-ended survey questions, transcribed interviews and speeches, wherever, in fact, the researcher is confronted with data in the form of natural language texts of social scientific interest. These methods are often used in exploratory data analysis, but can also be applied systematically with moderate statistical rigour in the development and testing of hypotheses at various theoretical levels, ranging from the statistics of word usage to changes within or between discourses over time. The general approach is in the tradition of content analysis, by which words which occur together in relatively close proximity in the same context are interpreted as relating to a common theme or concept in the discourse studied. We review a comprehensive set of tools to identify and visualize structures of co-occurrence of words and concepts both within, and in comparing, a number of texts. These produce results not essentially different from those reached by representing word co-occurrences in terms of network analysis or neural network programming using schematic linguistic templates of various kinds. A comparison of the relational data analysis vs. a dictionary-based MDS approach shows that these provide very close if not identical results, despite the fact that the underlying assumptions are frequently represented as different theoretical approaches.

Two biomedical sublanguages: a description based on the theories of Zellig Harris

Natural language processing (NLP) systems have been developed to provide access to the tremendous body of data and knowledge that is available in the biomedical domain in the form of natural language text. These NLP systems are valuable because they can encode and amass the information in the text so that it can be used by other automated processes to improve patient care and our understanding of disease processes and treatments. Zellig Harris proposed a theory of sublanguage that laid the foundation for natural language processing in specialized domains. He hypothesized that the informational content and structure form a specialized language that can be delineated in the form of a sublanguage grammar. The grammar can then be used by a language processor to capture and encode the salient information and relations in text. In this paper, we briefly summarize his language and sublanguage theories. In addition, we summarize our prior research, which is associated with the sublanguage grammars we developed for two different biomedical domains. These grammars illustrate how Harris’ theories provide a basis for the development of language processing systems in the biomedical domain. The two domains and their associated sublanguages discussed are: the clinical domain, where the text consists of patient reports, and the biomolecular domain, where the text consists of complete journal articles.