LSTM-CRF Model Research Articles

Integration Construction of Teaching Resource Library for Art and Design Majors under Interactivity Orientation

Abstract Building a well-rounded teaching resource platform for art design majors is essential for improving the quality and accessibility of educational materials. In this research, we used an innovative method, the LSTM-CRF model, to better understand what learners need and match these needs with relevant teaching materials. We also developed a new way to evaluate the effectiveness of our teaching resources, using an intelligent algorithm to find out how good the content really is. Our findings show that our resources score highly in areas like the quality of content, how often it’s updated, how personalized it is, and how well it’s promoted, with scores of 4.29, 4.24, 4.24, and 4.25, respectively. Interestingly, we found that student satisfaction varies significantly by their year in school and where they come from. This research not only sets the stage for future advancements in teaching resources for art and design but also suggests ways to make these resources even better.

The Construction of a Shared Resource Base for Teaching Chinese Culture under the Architecture of Disciplinary Knowledge Mapping

Abstract This paper proposes to use knowledge mapping to construct a shared resource base for Chinese culture teaching to realize the optimization and innovation of Chinese culture teaching. The internal data and external data of the Chinese teaching resource base are obtained through two ways textbook informatization and automatic semantic annotation. Then the data are preprocessed to ensure the accuracy of the data in the resource base. On this basis, the LSTM-CRF model is used to identify the entity naming of the knowledge points of Chinese teaching resources. Then, the named knowledge points of Chinese culture teaching resources are integrated and stored to complete the systematic construction of the Chinese culture teaching resource base. The empirical analysis of the proposed resource base is carried out. The data present that the average oral score of the students in the photo group is 37.2583, while that of the students in the experimental group is 43.1465, and the difference between the average scores of the two classes is about 5 points, which is a large score difference. The two-tailed significance sig value is 0.005 if equal variances are used, and the two-tailed significance sig value is 0.008 if equal variances are not used, both of which are less than 0.05, indicating that the method of this paper has a positive effect on improving the speaking scores of the students of the comprehensive Chinese culture class.

Research on the development of teaching resource library for art design majors based on artificial intelligence technology

Abstract This paper establishes the general architecture of the teaching resource repository through the design of course clusters and specific course resource organizations for the teaching characteristics of art and design majors and builds the business process from two aspects: teacher course resource archiving and student assignment management. For the association of knowledge points of teaching resources in the repository, the coding of text semantics is completed based on BERT, the text data is enhanced by a two-way maximum matching algorithm, and the enhanced data is input into the LSTM-CRF model for training to achieve entity prediction of knowledge points. For the recognition of art design teaching resources, the F1 score of this paper’s method is improved by 9.56% compared with the ATT-CNN model and 2.65% compared with the currently known better model ATT-BLSTM, which can better characterize the semantic information of the text.

Data Masking for Chinese Electronic Medical Records with Named Entity Recognition

With the rapid development of information technology, the electronification of medical records has gradually become a trend. In China, the population base is huge and the supporting medical institutions are numerous, so this reality drives the conversion of paper medical records to electronic medical records. Electronic medical records are the basis for establishing a smart hospital and an important guarantee for achieving medical intelligence, and the massive amount of electronic medical record data is also an important data set for conducting research in the medical field. However, electronic medical records contain a large amount of private patient information, which must be desensitized before they are used as open resources. Therefore, to solve the above problems, data masking for Chinese electronic medical records with named entity recognition is proposed in this paper. Firstly, the text is vectorized to satisfy the required format of the model input. Secondly, since the input sentences may have a long or short length and the relationship between sentences in context is not negligible. To this end, a neural network model for named entity recognition based on bidirectional long short-term memory (BiLSTM) with conditional random fields (CRF) is constructed. Finally, the data masking operation is performed based on the named entity recognition results, mainly using regular expression filtering encryption and principal component analysis (PCA) word vector compression and replacement. In addition, comparison experiments with the hidden markov model (HMM) model, LSTM-CRF model, and BiLSTM model are conducted in this paper. The experimental results show that the method used in this paper achieves 92.72% Accuracy, 92.30% Recall, and 92.51% F1_score, which has higher accuracy compared with other models.

An Automatic Error Detection Method for Engineering English Translation Based on the Deep Learning Model

Accuracy of deep learning model translation is a key index to evaluate the application performance of engineering English translation. In this paper, an automatic error detection system for English translation is proposed. In the particular task of grammar detection, researchers have gradually shifted their attention from statistical methods to neural network methods. Three deep learning algorithm models are established, and the multitask performance of the model is better than that of the conditional random field model and the LSTM-CRF model. The reason is that the multitask learning model of auxiliary tasks is included to some extent, which solves the problem of data sparsity and enables the model to be fully trained even under the condition of uneven label distribution. Thus, it performs better than other models in the task of syntax error detection. It realizes the word spelling error check based on the dictionary and uses the thought of editing distance to prompt the word error found, which can automatically check a large number of translations. On the basis of analyzing the sentence structure characteristics of engineering English translation, this paper realizes the detection of subject-verb agreement errors and analyzes the main word of the subject corresponding to the predicate verb by constructing the syntactic structure tree of the sentence, so as to realize the judgment of subject-verb agreement errors.

Construction of an Artificial Intelligence Writing Model for English Based on Fusion Neural Network Model.

This paper presents an in-depth study and analysis of the model of English writing using artificial intelligence algorithms of neural networks. Based on word vectors, the unsupervised disambiguation, and clustering of multimedia contexts extracted from massive online videos, the disambiguation accuracy reaches over 0.7, and the resulting small-scale multimedia context set can cover up to 90% of vocabulary learning tasks; user experiments show that the multimedia context learning system based on this method can improve the effectiveness and experience of ESL vocabulary learning, as well as the long-term word sense memory of learners. The results are 30% better. Based on the dependency grammatical relations and semantic metrics of collocations on a large-scale professional corpus, we established a collocation intention description and retrieval method in line with users' linguistic cognition and doubled the usage rate of collocation retrieval on the actual deployment system after half a year, becoming a user “sticky” ESL writing aid, and further defined style. Dictionaries only provide basic lexical definitions, and, even if supported by example sentences, they still cannot meet the needs of ESL authors in terms of expressive accuracy and richness. However, the current machine translation is based on the black box deep neural network construction, and its translation process is not understandable and interactive. Among the three algorithmic models constructed in this paper, the multitask learning model outperforms the conditional random field model and the LSTM-CRF model because the multitask learning model with auxiliary tasks solves the problem of sparse data to a certain extent, allowing the model to be trained more adequately in the case of uneven label distribution, and thus performs better than other models in the task of grammatical error detection.

Towards Tokenization and Part-of-Speech Tagging for Khmer: Data and Discussion

As a highly analytic language, Khmer has considerable ambiguities in tokenization and part-of-speech (POS) tagging processing. This topic is investigated in this study. Specifically, a 20,000-sentence Khmer corpus with manual tokenization and POS-tagging annotation is released after a series of work over the last 4 years. This is the largest morphologically annotated Khmer dataset as of 2020, when this article was prepared. Based on the annotated data, experiments were conducted to establish a comprehensive benchmark on the automatic processing of tokenization and POS-tagging for Khmer. Specifically, a support vector machine, a conditional random field (CRF) , a long short-term memory (LSTM) -based recurrent neural network, and an integrated LSTM-CRF model have been investigated and discussed. As a primary conclusion, processing at morpheme-level is satisfactory for the provided data. However, it is intrinsically difficult to identify further grammatical constituents of compounds or phrases because of the complex analytic features of the language. Syntactic annotation and automatic parsing for Khmer will be scheduled in the near future.

Research on Named Entity Recognition Method Based on Improved LSTM-CRF Model

Because the computer cannot directly understand the text corpus in the NLP task, the first thing to do is to represent the characteristics of the natural language numerically, and the word vector technology provides a good way to express it. Because Word2vec considers context and has fewer dimensions, it is now more popular words embedded. However, due to the particularity of Chinese, word2vec cannot accurately identify the polysemy of words. In this paper, a lightweight and effective method is used to merge vocabulary into character representation. This approach avoids designing complex sequence modeling architectures. for any neural network model, simply fine-tuning the character input layer can introduce vocabulary information. The model also uses the modified LSTM to bridge the enormous LSTM and the Transformer model. The interaction between input and context provides a richer modeling space that significantly improves testing on all four public datasets.

English Grammar Detection Based on LSTM-CRF Machine Learning Model.

Deep learning and neural network have been widely used in the field of speech, vocabulary, text, pictures, and other information processing fields, which has achieved excellent research results. Neural network algorithm and prediction model were used in this paper for the study and exploration of English grammar. Aiming at the application requirements of English grammar accuracy and standardization, we proposed a machine learning model based on LSTM-CRF to detect and analyze English grammar. This paper briefly summarized the development trend of deep learning and neural network algorithm and designed the structure pattern of radial basis function neural network in grammar semantic detection and analysis on the basis of deep learning artificial neural network theory. Based on the morphological features of English grammar, a grammar database was established according to the rules of English word segmentation. In this paper, we proposed an improved conditional random field CRF (Conditional Random Field) network model based on LSTM (Long Short-Term Memory) neural network. It can improve the problem that the traditional machine learning model relies on feature point selection in English grammar detection. The machine learning model based on LSTM-CRF was used to recognize English grammar text entities. The results show that the English grammar detection system based on the LSTM-CRF model can simplify the process structure in the recognition process, reduce the unnecessary operation cycle, and improve the overall detection accuracy.

Character-level neural network model based on Nadam optimization and its application in clinical concept extraction

Clinical concept extraction aims to quickly and effectively extract available data from complex and diverse clinical information, which is a crucial task for medical diagnosis using electronic medical records. Named entity recognition (NER) accurately marks essential information in clinical records based on the characteristics of the target entity, providing a way to extract clinical concepts. In the clinical concept extraction task, the existing methods are not satisfactory to obtain accurate labelling results in the face of large-scale and complex clinical information. To solve this problem, we improve and optimize a named entity recognition method based on the LSTM-CRF model. First, the improved deep neural network model uses two optional configurations of Convolutional Neural Network (CNN) and Bidirectional Long Short-Term Memory (BLSTM) to achieve character-level representation. Then the BLSTM layer obtains the context information of the target word, and the Conditional Random Field (CRF) gives constraints to ensure the standardization of the label. On this basis, Nadam is used to optimize the training process of the network. The experimental results show that our new method has an F1 score of 84.61 on the public dataset of 2010 i2b2/VA concept extraction task, which exceeds the LSTM-CRF model. And the recall of 85.41 is ahead of all the methods evaluated on this data set.

Pretraining to Recognize PICO Elements from Randomized Controlled Trial Literature.

PICO (Population/problem, Intervention, Comparison, and Outcome) is widely adopted for formulating clinical questions to retrieve evidence from the literature. It plays a crucial role in Evidence-Based Medicine (EBM). This paper contributes a scalable deep learning method to extract PICO statements from RCT articles. It was trained on a small set of richly annotated PubMed abstracts using an LSTM-CRF model. By initializing our model with pretrained parameters from a large related corpus, we improved the model performance significantly with a minimal feature set. Our method has advantages in minimizing the need for laborious feature handcrafting and in avoiding the need for large shared annotated data by reusing related corpora in pretraining with a deep neural network.

Word Embedding에 PCA를 적용한 개체명 인식 모델을 위한 효율적인 학습방법 연구

The Bidirectional LSTM CRF model used for Named Entity Recognition takes much time to train Named Entity. The hyper-parameters of Word Embedding used as input data in this model affect performance and training time. However, there is very little research on the number of dimensions, which is one of the parameters of Word Embedding. In this paper, we obtain proper number of 4-Word Embeddings (fastText, GloVe, skipgram, CBOW) considering performance and training time in Bidirectional LSTM CRF which can input large amount of data. Next, apply the PCA to the word vector in Word Embedding to reduce the dimension to small dimensional (10 dimensions) intervals. Therefore, applying PCA to conventional Word Embedding and training Word Embedding with small dimensional intervals shows that the model can be trained by maintaining or improving performance based on stable training time in fewer dimensions than conventional Word Embedding.

Forward-Looking Element Recognition Based on the LSTM-CRF Model with the Integrity Algorithm

A state-of-the-art entity recognition system relies on deep learning under data-driven conditions. In this paper, we combine deep learning with linguistic features and propose the long short-term memory-conditional random field model (LSTM-CRF model) with the integrity algorithm. This approach is primarily based on the use of part-of-speech (POS) syntactic rules to correct the boundaries of LSTM-CRF model annotations and improve its performance by raising the integrity of the elements. The method incorporates the advantages of the data-driven method and dependency syntax, and improves the precision rate of the elements without losing recall rate. Experiments show that the integrity algorithm is not only easy to combine with the other neural network model, but the overall effect is better than several advanced methods. In addition, we conducted cross-domain experiments based on a multi-industry corpus in the financial field. The results indicate that the method can be applied to other industries.

Resource Mention Extraction for MOOC Discussion Forums

In discussions hosted on discussion forums for massive online open courses (MOOCs), references to online learning resources are often of central importance. They contextualize the discussion, anchoring the discussion participants' presentation of the issues and their understanding. However, they are usually mentioned in free text, without appropriate hyperlinking to their associated resource. Automated learning resource mention hyperlinking and categorization will facilitate discussion and searching within the MOOC forums and also benefit the contextualization of such resources across disparate views. We propose the novel problem of learning resource mention identification in MOOC forums, i.e., to identify resource mentions in discussions and classify them into pre-defined resource types. As this is a novel task with no publicly available data, we first contribute a large-scale labeled dataset-dubbed the forum resource mention (FoRM) dataset-to facilitate our current research and future research on this task. The FoRM contains over 10 000 real-world forum threads in collaboration with Coursera, with more than 23 000 manually labeled resource mentions. We then formulate this task as a sequence tagging problem and investigate solution architectures to address the problem. Importantly, we identify two major challenges that hinder the applications of sequence tagging models to the task: (1) the diversity of resource mention expression and (2) long-range contextual dependencies. We address these challenges by incorporating character-level and thread context information into an LSTM-CRF model. First, we incorporate a character encoder to address the out-of-vocabulary problem caused by the diversity of mention expressions. Second, to address the context dependency challenge, we encode thread contexts using an RNN-based context encoder and apply the attention mechanism to selectively leverage useful context information during sequence tagging. The experiments on FoRM show that the proposed method improves the baseline deep sequence tagging models notably, significantly bettering performance on instances that exemplify two challenges.

Expert Information Automatic Extraction for IOT Knowledge Base

With the rapid development of IOT technology, the requirement of effective and accurate retrieval of domain knowledge is growing. Automatically extract various information of expert from the massive web pages and generate a dynamic and wholeness profile model are important for knowledge base. However, the obvious differences in structure and content semantics of web pages between any two websites shows traditional web crawler are hard to understand the semantic of the web page and extract the critical information of expert. Therefore, a six-dimension expert profile model was introduced and then a sequence tagging method with LSTM-CRF model was proposed to automatically extract rich semantic information basing on organization structure, meaning of words and attributes of experts. The results of the experiment on test data sets illustrated that the precision rate and recall rate about the job experience and research field of experts are 67.8%, 66.6% and 82.4%, 79.6%, respectively. In addition, the overall average F value about some obvious features of expert, such as name, title, email, achievement, etc., reaches 82.5%, which is better than the results by MEMM and LSTM algorithms.

Adversarial Learning for Chinese NER From Crowd Annotations

To quickly obtain new labeled data, we can choose crowdsourcing as an alternative way at lower cost in a short time. But as an exchange, crowd annotations from non-experts may be of lower quality than those from experts. In this paper, we propose an approach to performing crowd annotation learning for Chinese Named Entity Recognition (NER) to make full use of the noisy sequence labels from multiple annotators. Inspired by adversarial learning, our approach uses a common Bi-LSTM and a private Bi-LSTM for representing annotator-generic and -specific information. The annotator-generic information is the common knowledge for entities easily mastered by the crowd. Finally, we build our Chinese NE tagger based on the LSTM-CRF model. In our experiments, we create two data sets for Chinese NER tasks from two domains. The experimental results show that our system achieves better scores than strong baseline systems.

Recurrent neural networks with specialized word embeddings for health-domain named-entity recognition

BackgroundPrevious state-of-the-art systems on Drug Name Recognition (DNR) and Clinical Concept Extraction (CCE) have focused on a combination of text “feature engineering” and conventional machine learning algorithms such as conditional random fields and support vector machines. However, developing good features is inherently heavily time-consuming. Conversely, more modern machine learning approaches such as recurrent neural networks (RNNs) have proved capable of automatically learning effective features from either random assignments or automated word “embeddings”. Objectives(i) To create a highly accurate DNR and CCE system that avoids conventional, time-consuming feature engineering. (ii) To create richer, more specialized word embeddings by using health domain datasets such as MIMIC-III. (iii) To evaluate our systems over three contemporary datasets. MethodsTwo deep learning methods, namely the Bidirectional LSTM and the Bidirectional LSTM-CRF, are evaluated. A CRF model is set as the baseline to compare the deep learning systems to a traditional machine learning approach. The same features are used for all the models. ResultsWe have obtained the best results with the Bidirectional LSTM-CRF model, which has outperformed all previously proposed systems. The specialized embeddings have helped to cover unusual words in DrugBank and MedLine, but not in the i2b2/VA dataset. ConclusionsWe present a state-of-the-art system for DNR and CCE. Automated word embeddings has allowed us to avoid costly feature engineering and achieve higher accuracy. Nevertheless, the embeddings need to be retrained over datasets that are adequate for the domain, in order to adequately cover the domain-specific vocabulary.