Named Entity Extraction Research Articles

Clinical named entity extraction for extracting information from medical data

Clinical named entity extraction (NER) based on deep learning gained much attention among researchers and data analysts. This paper proposes a NER approach to extract valuable Parkinson’s disease-related information. To develop an effective NER method and to handle problems in disease data analytics, a unique NER technique applies a “recognize-map-extract (RME)” mechanism and aims to deal with complex relationships present in the data. Due to the fast-growing medical data, there is a challenge in the development of suitable deep-learning methods for NER. Furthermore, the traditional machine learning approaches rely on the time-consuming process of creating corpora and cannot extract information for specific needs and locations in certain situations. This paper presents a clinical NER approach based on a convolutional neural network (CNN) for better use of specific features around medical entities and analyzes the performance of the proposed approach through fine-tuning NER with effective pre-training on the BC5CDR dataset. The proposed method uses annotation of entities for various medical concepts. The second stage develops a clinically NER method. This proposed method shows interesting results on the performance measures achieving a precision of 92.57%, recall of 92.22%, and F1- measure of 91.6%.

Extracting Geoscientific Dataset Names from the Literature Based on the Hierarchical Temporal Memory Model

Extracting geoscientific dataset names from the literature is crucial for building a literature–data association network, which can help readers access the data quickly through the Internet. However, the existing named-entity extraction methods have low accuracy in extracting geoscientific dataset names from unstructured text because geoscientific dataset names are a complex combination of multiple elements, such as geospatial coverage, temporal coverage, scale or resolution, theme content, and version. This paper proposes a new method based on the hierarchical temporal memory (HTM) model, a brain-inspired neural network with superior performance in high-level cognitive tasks, to accurately extract geoscientific dataset names from unstructured text. First, a word-encoding method based on the Unicode values of characters for the HTM model was proposed. Then, over 12,000 dataset names were collected from geoscience data-sharing websites and encoded into binary vectors to train the HTM model. We conceived a new classifier scheme for the HTM model that decodes the predictive vector for the encoder of the next word so that the similarity of the encoders of the predictive next word and the real next word can be computed. If the similarity is greater than a specified threshold, the real next word can be regarded as part of the name, and a successive word set forms the full geoscientific dataset name. We used the trained HTM model to extract geoscientific dataset names from 100 papers. Our method achieved an F1-score of 0.727, outperforming the GPT-4- and Claude-3-based few-shot learning (FSL) method, with F1-scores of 0.698 and 0.72, respectively.

A case study in applying artificial intelligence-based named entity recognition to develop an automated ophthalmic disease registry

PurposeAdvances in artificial intelligence (AI)-based named entity extraction (NER) have improved the ability to extract diagnostic entities from unstructured, narrative, free-text data in electronic health records. However, there is a lack of ready-to-use tools and workflows to encourage the use among clinicians who often lack experience and training in AI. We sought to demonstrate a case study for developing an automated registry of ophthalmic diseases accompanied by a ready-to-use low-code tool for clinicians.MethodsWe extracted deidentified electronic clinical records from a single centre’s adult outpatient ophthalmology clinic from November 2019 to May 2022. We used a low-code annotation software tool (Prodigy) to annotate diagnoses and train a bespoke spaCy NER model to extract diagnoses and create an ophthalmic disease registry.ResultsA total of 123,194 diagnostic entities were extracted from 33,455 clinical records. After decapitalisation and removal of non-alphanumeric characters, there were 5070 distinct extracted diagnostic entities. The NER model achieved a precision of 0.8157, recall of 0.8099, and F score of 0.8128.ConclusionWe presented a case study using low-code artificial intelligence-based NLP tools to produce an automated ophthalmic disease registry. The workflow created a NER model with a moderate overall ability to extract diagnoses from free-text electronic clinical records. We have produced a ready-to-use tool for clinicians to implement this low-code workflow in their institutions and encourage the uptake of artificial intelligence methods for case finding in electronic health records.

Discrete sequence rearrangement based self-supervised chinese named entity recognition for robot instruction parsing

Named entity recognition (NER) plays an important role in information extraction tasks, but most models rely on large-scale labeled data. Getting the model to move away from large-scale labeled datasets is challenging. In this paper, a SCNER (Self-Supervised NER) model is proposed. The BiLSTM (Bidirectional LSTM) is adopted as the named entity extractor, and an Instruction Generation Subsystem (IGS) is proposed to generate "Retelling Instructions", which analyzes the similarities between the input instructions and "Retelling Instructions" as the losses for model training. A series of rules based on traditional learning rules have been proposed for discrete forward computation and error backpropagation. It mimics language learning in human infants and constructs a SCNER model. This model is used for robot instruction understanding and can be trained on unlabeled datasets to extract named entities from instructions. Experimental results show that the proposed model is competitive with the supervised BiLSTM-CRF and BERT-NER models. In addition, the model is applied to a real robot, which verifies the practicality of SCNER.

IntelliExtract: An End-to-End Framework for Chinese Resume Information Extraction from Document Images

Traditional document processing can be labor-intensive and time-consuming to manually extract and organize the information in a document. This manual process is often inefficient and error-prone. In order to improve processing efficiency and accuracy of document data, we develop IntelliExtract, an end-to-end framework designed for document information extraction. This is a comprehensive framework that includes image text detection and recognition, information extraction, and document intelligent question-answering. Some recent models and algorithms are employed, OCR models for converting scanned documents into machine readable text, layout analysis algorithms for understanding the spatial arrangement of document elements, and information extraction techniques for extracting structured data from unstructured documents. To evaluate the effectiveness of the framework, we conducted experiments by employing a Chinese Talent Resumes Dataset for visualizing the results. For named entity extraction, the confidence level of the extracted results from the text in the images is generally above 0.95. The proposed framework provides a powerful tool for enterprises, educational institutions, and other entities in processing document information, and holds promise for significant practical applications.

A Study on a Knowledge Graph Construction Method of Safety Reports for Process Industries

There are some representative reports in industrial safety engineering, such as the Hazard and Operability Analysis and Pre-Hazard Analysis; however, a large amount of industrial safety knowledge in the report has not been fully explored. In order to reuse and release the value of industrial safety knowledge, this paper constructs a new industrial safety knowledge extraction framework. The framework combines the asset management shell to summarize the knowledge concept entities of machine description language and model description language. According to the safety report template, the framework also constructs a new industrial safety knowledge-mapping standard structure. Specifically, firstly, considering that the knowledge structure of safety reports is different in different processes of the process industry, this paper innovatively proposes a general industrial safety knowledge-mapping standard structure, which provides a practical solution for the integration of industrial knowledge representation problems in different processes. Secondly, based on the research progress of named entities, this paper presents an industrial named entity extraction method (INERM) for the process industry. This method designs an entity weight model to calculate the entity weight of each sentence, and adds part-of-speech weight to improve the entity extraction algorithm, which alleviates the problem that the existing entity extraction methods cannot reasonably use the semantic information and context of word. Finally, we construct a triple of industrial safety knowledge based on the rules and store it in Neo4j. In this paper, four semantic-type templates and five semantic relation templates are constructed based on the new industrial safety knowledge map standardization construction process of the process industry. The comparative experiments show that the accuracy of the INERM on the test set is improved by 17 percentage points on average compared with other key entity extraction algorithms. A total of 1329 entities are constructed in the directional application example of the fluid transportation process, which provides a large number of references for the safety of the fluid transportation process and is more conducive to improving the safety guarantee of the fluid transport process.

Method and Models of Extraction of Knowledge from Medical Documents

The paper analyzes the problem of extracting knowledge from clinical recommendations presented in the form of semi-structured corpora of text documents in natural language, taking into account their periodic updating. The considered methods of intellectual analysis of the accumulated arrays of medical data make it possible to automate a number of tasks aimed at improving the quality of medical care due to significant decision support in the treatment process. A brief review of well-known publications has been made, highlighting approaches to automating the construction of ontologies and knowledge graphs in the problems of semantic modeling of a problem-oriented text corpus. The structural and functional organization of the system of knowledge extraction and automatic construction of an ontology and a knowledge graph of a problem-oriented corpus for a specific subject area is presented. The main stages of knowledge extraction and dynamic updating of the knowledge graph are considered: named entity extraction, semantic annotation, term and keyword extraction, topic modeling, topic identification, and relationship extraction. The formalized representation of texts was obtained using a pre-trained BERT transformer model. The automatic selection of triplets "object" - "action" - "subject" based on part-of-speech markup of the text corpus was used to construct fragments of the knowledge graph. An experiment was carried out on a corpus of medical texts on a given topic (162 documents of depersonalized case histories of patients of a pediatric center) without preliminary markup in order to test the proposed solution for extracting triplets and constructing a knowledge graph based on them. An analysis of the experimental results confirms the need for a deeper markup of the corpus of text documents to take into account the specifics of medical text documents. For an unmarked corpus of texts, the proposed solution demonstrates satisfactory performance in view of the selection of atomic fragments included in the automatically generated ontology.

Comparing the variability of ingredient, strength, and dose form information from electronic prescriptions with RxNorm drug product descriptions.

To determine the variability of ingredient, strength, and dose form information from drug product descriptions in real-world electronic prescription (e-prescription) data. A sample of 10399324 e-prescriptions from 2019 to 2021 were obtained. Drug product descriptions were analyzed with a named entity extraction model and National Drug Codes (NDCs) were used to get RxNorm Concept Unique Identifiers (RxCUI) via RxNorm. The number of drug product description variants for each RxCUI was determined. Variants identified were compared to RxNorm to determine the extent of matching terminology used. A total of 353002 unique pairs of drug product descriptions and NDCs were analyzed. The median (1st-3rd quartile) number of variants extracted for each standardized expression in RxNorm, was 3 (2-7) for ingredients, 4 (2-8) for strength, and 41 (11-122) for dosage forms. Of the pairs, 42.35% of ingredients (n = 328032), 51.23% of strengths (n = 321706), and 10.60% of dose forms (n = 326653) used matching terminology, while 16.31%, 24.85%, and 13.05% contained nonmatching terminology, respectively. A wide variety of drug product descriptions makes it difficult to determine whether 2 drug product descriptions describe the same drug product (eg, using abbreviations to describe an active ingredient or using different units to represent a concentration). This results in patient safety risks that lead to incorrect drug products being ordered, dispensed, and used by patients. Implementation and use of standardized terminology may reduce these risks. Drug product descriptions on real-world e-prescriptions exhibit large variation resulting in unnecessary ambiguity and potential patient safety risks.

Extracting clinical named entity for pituitary adenomas from Chinese electronic medical records

ObjectivePituitary adenomas are the most common type of pituitary disorders, which usually occur in young adults and often affect the patient’s physical development, labor capacity and fertility. Clinical free texts noted in electronic medical records (EMRs) of pituitary adenomas patients contain abundant diagnosis and treatment information. However, this information has not been well utilized because of the challenge to extract information from unstructured clinical texts. This study aims to enable machines to intelligently process clinical information, and automatically extract clinical named entity for pituitary adenomas from Chinese EMRs.MethodsThe clinical corpus used in this study was from one pituitary adenomas neurosurgery treatment center of a 3A hospital in China. Four types of fine-grained texts of clinical records were selected, which included notes from present illness, past medical history, case characteristics and family history of 500 pituitary adenoma inpatients. The dictionary-based matching, conditional random fields (CRF), bidirectional long short-term memory with CRF (BiLSTM-CRF), and bidirectional encoder representations from transformers with BiLSTM-CRF (BERT-BiLSTM-CRF) were used to extract clinical entities from a Chinese EMRs corpus. A comprehensive dictionary was constructed based on open source vocabularies and a domain dictionary for pituitary adenomas to conduct the dictionary-based matching method. We selected features such as part of speech, radical, document type, and the position of characters to train the CRF-based model. Random character embeddings and the character embeddings pretrained by BERT were used respectively as the input features for the BiLSTM-CRF model and the BERT-BiLSTM-CRF model. Both strict metric and relaxed metric were used to evaluate the performance of these methods.ResultsExperimental results demonstrated that the deep learning and other machine learning methods were able to automatically extract clinical named entities, including symptoms, body regions, diseases, family histories, surgeries, medications, and disease courses of pituitary adenomas from Chinese EMRs. With regard to overall performance, BERT-BiLSTM-CRF has the highest strict F1 value of 91.27% and the highest relaxed F1 value of 95.57% respectively. Additional evaluations showed that BERT-BiLSTM-CRF performed best in almost all entity recognition except surgery and disease course. BiLSTM-CRF performed best in disease course entity recognition, and performed as well as the CRF model for part of speech, radical and document type features, with both strict and relaxed F1 value reaching 96.48%. The CRF model with part of speech, radical and document type features performed best in surgery entity recognition with relaxed F1 value of 95.29%.ConclusionsIn this study, we conducted four entity recognition methods for pituitary adenomas based on Chinese EMRs. It demonstrates that the deep learning methods can effectively extract various types of clinical entities with satisfying performance. This study contributed to the clinical named entity extraction from Chinese neurosurgical EMRs. The findings could also assist in information extraction in other Chinese medical texts.

An Entity Recognition Model Based on Deep Learning Fusion of Text Feature

Nowadays a large amount of knowledge has been born on the Internet and the way of constructing knowledge graph is not uniform. Due to the recent outbreak of numerous diseases, the community has placed more importance on the healthcare system. Diabetes is a severe disease that affect people's health. To assist the health sector in combating this deadly disease, the authors developed a deep learning strategy for diabetes named entity extraction based on a fusion of text characteristic and relationship extraction utilizing text data as the object. This study aims to develop a multi-feature entity recognition model that considers the differences in text features across different fields. Firstly, in the word embedding layer, a multi-feature word embedding algorithm is proposed, which integrates Pinyin, radical, and the meaning of the character itself, so that the word embedding vector has the characteristics of Chinese characters and diabetes text. Then in modeling, CNN and BiLSTM are used to extract the local and global features before and after the text sequence, respectively, which solved the problem that the traditional method cannot capture the dependence before and after the text sequence. Finally, CRF is used to output the predicted tag sequence. The experimental results show that the multi-feature embedding algorithm and local features extracted by CNN can effectively improve the recognition effect of the entity recognition model.

Development of Dialogue Management System for Banking Services

Rapid increase in conversational AI and user chat data lead to intensive development of dialogue management systems (DMS) for various industries. Yet, for low-resource languages, such as Azerbaijani, very little research has been conducted. The main purpose of this work is to experiment with various DMS pipeline set-ups to decide on the most appropriate natural language understanding and dialogue manager settings. In our project, we designed and evaluated different DMS pipelines with respect to the conversational text data obtained from one of the leading retail banks in Azerbaijan. In the work, the main two components of DMS—Natural language Understanding (NLU) and Dialogue Manager—have been investigated. In the first step of NLU, we utilized a language identification (LI) component for language detection. We investigated both built-in LI methods such as fastText and custom machine learning (ML) models trained on the domain-based dataset. The second step of the work was a comparison of the classic ML classifiers (logistic regression, neural networks, and SVM) and Dual Intent and Entity Transformer (DIET) architecture for user intention detection. In these experiments we used different combinations of feature extractors such as CountVectorizer, Term Frequency-Inverse Document Frequency (TF-IDF) Vectorizer, and word embeddings for both word and character n-gram based tokens. To extract important information from the text messages, Named Entity Extraction (NER) component was added to the pipeline. The best NER model was chosen among conditional random fields (CRF) tagger, deep neural networks (DNN), models and build in entity extraction component inside DIET architecture. Obtained entity tags fed to the Dialogue Management module as features. All NLU set-ups were followed by the Dialogue Management module that contains a Rule-based Policy to handle FAQs and chitchats as well as a Transformer Embedding Dialogue (TED) Policy to handle more complex and unexpected dialogue inputs. As a result, we suggest a DMS pipeline for a financial assistant, which is capable of identifying intentions, named entities, and a language of text followed by policies that allow generating a proper response (based on the designed dialogues) and suggesting the best next action.

An Efficient Cross-Lingual BERT Model for Text Classification and Named Entity Extraction in Multilingual Dataset

In recent times, with the rise of the internet, everyone is being bombarded with tons of information and data from various sources like websites, blogs and articles, social media posts and comments, e-news portals etc. Now all these data are mostly unstructured. In this paper, the authors have tried to explore the efficiency of the cross-lingual BERT model i.e. M-BERT for text classification and named entity extraction on multilingual data. The authors have used datasets of three different languages namely: French, German and Portuguese to evaluate the model performance.

A deep-learning model for semantic role labelling in medical documents

We built a model labelling the Predicate Argument Structure (PAS) for biomedical documents. PAS is an important semantic information of any document, because it reveals the main event mentioned in each sentence. Extracting PAS in a sentence is an important premise for the computer to solve a series of other problems related to the semantics in text such as event extraction, named entity extraction, question answering system… The predicate argument structure is domain dependent. Therefore, in Biomedical field, it is required to define a completely new Predicate Argument frame compared to the general field. For a machine learning model to work well with a new argument frame, identifying a new feature set is required. This is difficult, manual and requires a lot of expert labor. To address this challenge, we chose to train our model with Deep Learning method utilizing Bi-directional Long Short Term Memory. Deep learning is a machine learning method that does not require defining the feature sets manually. In addition, we also integrate Highway Connection between hidden neuron layers to minimize derivative loss. Besides, to overcome the problem of small training corpus, we integrate Deep Learning with Multi-task Learning technique. Multi-task Learning helps the main task (PAS tagging) to be complemented with knowledge learnt from a closely related task, the NER. Our model achieved F1 = 75.13% without any manually designed feature, thereby showing the prospect of Deep Learning in this domain. In addition, the experiment results also show that Multi-task Learning is an appropriate technique to overcome the problem of little training data in biomedical fields, by improving the F1 score.

Information Extraction Tasks based on BERT and SpaCy on Tourism Domain

In this paper, we present two methodologies to extract particular information based on the full text returned from the search engine to facilitate the users. The approaches are based three tasks: name entity recognition (NER), text classification and text summarization. The first step is the building training data and data cleansing. We consider tourism domain such as restaurant, hotels, shopping and tourism data set crawling from the websites. First, the tourism data are gathered and the vocabularies are built. Several minor steps include sentence extraction, relation and name entity extraction for tagging purpose. These steps are needed for creating proper training data. Then, the recognition model of a given entity type can be built. From the experiments, given review texts, we demonstrate to build the model to extract the desired entity,i.e, name, location, facility as well as relation type, classify the reviews or summarize the reviews. Two tools, SpaCy and BERT, are used to compare the performance of these tasks.

Automatic Arabic Named Entity Extraction and Classification for Information Retrieval

This article tries to explain our rule-based Arabic Named Entity recognition (NER) and classification system. It is based on lists of classified proper names (PN) and particularly on syntactico-semantic patterns resulting in fine classification of Arabic NE. These patterns use syntactico-semantic combination of morpho-syntactic and syntactic entities. It also uses lexical classification of trigger words and NE extensions. These linguistic data are essential not only to name entity extraction but also to the taxonomic classification and to determining the NE frontiers. Our method is also based on the contextualisation and on the notion of NE class attributes and values. Inspired from X-bar theory and immediate constituents, we built a rule-based NER system composed of five levels of syntactico-semantic combination. We also show how the fine NE annotations in our system output (XML database) is exploited in information retrieval and information extraction.

CoMNRank: An integrated approach to extract and prioritize human microbial metabolites from MEDLINE records.

Trillions of bacteria in human body (human microbiota) affect human health and diseases by controlling host functions through small molecule metabolites.An accurate and comprehensive catalog of the metabolic output from human microbiota is critical for our deep understanding of how microbial metabolism contributes to human health.The large number of published biomedical research articles is a rich resource of microbiome studies.However, automatically extracting microbial metabolites from free-text documents and differentiating them from other human metabolites is a challenging task.Here we developed an integrated approach called Co-occurrence Metabolite Network Ranking (CoMNRank) by combining named entity extraction, network construction and topic sensitive network-based prioritization to extract and prioritize microbial metabolites from biomedical articles. The text data included 28,851,232 MEDLINE records.CoMNRank consists of three steps: (1) extraction of human metabolites from MEDLINE records; (2) construction of a weighted co-occurrence metabolite network (CoMN); (3) prioritization and differentiation of microbial metabolites from other human metabolites. For the first step of CoMNRank, we extracted 11,846 human metabolites from MEDLINE articles, with a baseline performance of precision of 0.014, recall of 0.959 and F1 of 0.028.We then constructed a weighted CoMN of 6,996 nodes and 986,186 edges.CoMNRank effectively prioritized microbial metabolites: the precision of top ranked metabolites is 0.45, a 31-fold enrichment as compared to the overall precision of 0.014.Manual curation of top 100 metabolites showed a true precision of 0.67, among which 48% true positives are not captured by existing databases. Our study sets the foundation for future tasks of microbial entity and relationship extractions as well as data-driven studies of how microbial metabolism contributes to human health and diseases.

Extracting Time Expressions and Named Entities with Constituent-Based Tagging Schemes

Time expressions and named entities play important roles in data mining, information retrieval, and natural language processing. However, the conventional position-based tagging schemes (e.g., the BIO and BILOU schemes) that previous research used to model time expressions and named entities suffer from the problem of inconsistent tag assignment. To overcome the problem of inconsistent tag assignment, we designed a new type of tagging schemes to model time expressions and named entities based on their constituents. Specifically, to model time expressions, we defined a constituent-based tagging scheme termed TOMN scheme with four tags, namely T, O, M, and N, indicating the defined constituents of time expressions, namely time token, modifier, numeral, and the words outside time expressions. To model named entities, we defined a constituent-based tagging scheme termed UGTO scheme with four tags, namely U, G, T, and O, indicating the defined constituents of named entities, namely uncommon word, general modifier, trigger word, and the words outside named entities. In modeling, our TOMN and UGTO schemes model time expressions and named entities under conditional random fields with minimal features according to an in-depth analysis for the characteristics of time expressions and named entities. Experiments on diverse datasets demonstrate that our proposed methods perform equally with or more effectively than representative state-of-the-art methods on both time expression extraction and named entity extraction.

Deep neural networks for Arabic information extraction

PurposeThe purpose of this paper is to present a system based on deep neural networks to extract particular entities from natural language text, knowing that a massive amount of textual information is electronically available at present. Notably, a large amount of electronic text data indicates great difficulty in finding or extracting relevant information from them.Design/methodology/approachThis study presents an original system to extract Arabic-named entities by combining a deep neural network-based part-of-speech tagger and a neural network-based named entity extractor. Firstly, the system extracts the grammatical classes of the words with high precision depending on the context of the word. This module plays the role of the disambiguation process. Then, a second module is used to extract the named entities.FindingsUsing deep neural networks in natural language processing, requires tuning many hyperparameters, which is a time-consuming process. To deal with this problem, applying statistical methods like the Taguchi method is much requested. In this study, the system is successfully applied to the Arabic-named entities recognition, where accuracy of 96.81 per cent was reported, which is better than the state-of-the-art results.Research limitations/implicationsThe system is designed and trained for the Arabic language, but the architecture can be used for other languages.Practical implicationsInformation extraction systems are developed for different applications, such as analysing newspaper articles and databases for commercial, political and social objectives. Information extraction systems also can be built over an information retrieval (IR) system. The IR system eliminates irrelevant documents and paragraphs.Originality/valueThe proposed system can be regarded as the first attempt to use double deep neural networks to increase the accuracy. It also can be built over an IR system. The IR system eliminates irrelevant documents and paragraphs. This process reduces the mass number of documents from which the authors wish to extract the relevant information using an information extraction system.

Medical Named Entity Extraction from Chinese Resident Admit Notes Using Character and Word Attention-Enhanced Neural Network.

The resident admit notes (RANs) in electronic medical records (EMRs) is first-hand information to study the patient’s condition. Medical entity extraction of RANs is an important task to get disease information for medical decision-making. For Chinese electronic medical records, each medical entity contains not only word information but also rich character information. Effective combination of words and characters is very important for medical entity extraction. We propose a medical entity recognition model based on a character and word attention-enhanced (CWAE) neural network for Chinese RANs. In our model, word embeddings and character-based embeddings are obtained through character-enhanced word embedding (CWE) model and Convolutional Neural Network (CNN) model. Then attention mechanism combines the character-based embeddings and word embeddings together, which significantly improves the expression ability of words. The new word embeddings obtained by the attention mechanism are taken as the input to bidirectional long short-term memory (BI-LSTM) and conditional random field (CRF) to extract entities. We extracted nine types of key medical entities from Chinese RANs and evaluated our model. The proposed method was compared with two traditional machine learning methods CRF, support vector machine (SVM), and the related deep learning models. The result shows that our model has better performance, and the result of our model reaches 94.44% in the F1-score.

ВЫДЕЛЕНИЕ ИМЕНОВАННЫХ СУЩНОСТЕЙ ИЗ ТЕКСТОВ РАСПОРЯДИТЕЛЬНЫХ ДОКУМЕНТОВ С ПОМОЩЬЮ ГЛУБОКИХ НЕЙРОННЫХ СЕТЕЙ

Named Entity Extraction (NER) is the task of extracting information from text data that belongs to predefined categories, such as organizations names, place names, people's names, etc. Within the framework of the presented work, was developed an approach for the additional training of deep neural networks with the attention mechanism (BERT architecture). It is shown that the preliminary training of the language model in the tasks of recovering the masked word and determining the semantic relatedness of two sentences can significantly improve the quality of solving the problem of NER. One of the best results has been achieved in the task of extracting named entities on the RuREBus dataset. One of the key features of the described solution is the closeness of the formulation to real business problems and the selection of entities not of a general nature, but specific to the economic industry.