Conditional Random Field Model Research Articles

Segmentation of wheat scab fungus spores based on CRF_ResUNet++

Accurate segmentation of tiny spore pictures is a significant technique in computer-aided evaluation of wheat scab. A CRF_ResUNet++ spore segmentation model using UNet++ as the fundamental framework is proposed to address the issues of uneven brightness, low spore-background contrast, and spore adhesion in microscopically collected Fusarium spore images. ResNet and fully connected conditional random fields (CRF) are combined in this model to address these issues. First, a residual block ResNet is introduced into UNet++ to enhance the propagation of features and extract more spore detail information. Next, the spore image is initially segmented using UNet++ with an encoder-decoder structure. Finally, postprocessing is carried out using a fully connected conditional random field model to obtain more precise edges and complete spore regions. The laboratory collection dataset test results revealed that the F1-score and MIoU reached 0.943 and 0.925, respectively, with an average detection Precision improvement of 3.2 % over the original UNet++ model. In comparison to existing models, the suggested model can segment spore pictures in challenging circumstances, such as when they contain independent and inter-adherent spores, with excellent computer vision processing capabilities effects and objective evaluation metric segmentation effects. In conclusion, the method in this study can reliably detect and segment adherent wheat scab spores in complex backgrounds, which offers critical technical support for automatic detection of adherent wheat scab spores and early spore concentration prediction in complex environments in the field.

Deformation Gated Recurrent Network for Lane-Level Abnormal Driving Behavior Recognition

As a significant part of traffic accident prevention, abnormal driving behavior recognition has been receiving extensive attention. However, the granularity of existing abnormal driving behavior recognition is mostly at road-level, and these methods’ high complexity leads to high overhead on training and recognition. In this article, we propose a deformation gated recurrent network for lane-level abnormal driving behavior recognition. Firstly, we use conditional random field model to calculate the lane change necessity of the vehicle, which helps us to distinguish whether the lane-changing behavior is reasonable. Secondly, we propose deformation gated recurrent network (DF-GRN) and trajectory entropy to capture the implicit relationship between trajectories and shorten recognition time. Finally, we get classified results including aggressive, distracted and normal driving behavior from the network. Distracted and aggressive behavior will be marked as anomaly. The effectiveness and real-time nature of the network are verified by experiments on Hangzhou and Chengdu location datasets.

A Named Entity Recognition Approach for Electronic Medical Records Using BERT Semantic Enhancement and BiLSTM

Aiming at the problems of missing local context features, single word vector representation, and low entity recognition accuracy, a method for e-medical recording with named entity recognition, which is based on BERT and model fusion, is proposed. First, with the model of BERT for pre-training, the preceding and following contextual information is fused for the enhancement of word semantic representation and alleviation of the problem of polysemy; second, the network of bi-directional long-short term memory is for obtaining the sequence feature matrix, generation of optimal sequence in global sense achieved through the conditional random field model; finally, data enhancement is used to alleviate the class imbalance and improve the model ability in generalization. Results of the experiments find model proposal measured by F1 on CCKS21 data set reaches 0.8548, which is 0.51% and 0.08% higher than models with ID-CNNs-CRF and multi-task RNN. This demonstrates the excellent performance of the method proposed in this paper in improving named entity recognition.

A Convolutional Neural Network Model for Setswana Named Entity Recognition

Named entity recognition (NER) is a key component of the core task of natural language processing (NLP). In order to represent language, neural networks have been used starting in the 2000s, which enhanced entity recognition outcomes. The Setswana language, in contrast, has never been used with neural networks, in particular convolutional neural networks (CNN). Recently, problems with NLP have been addressed using CNNs, and the results have been quite interesting. CNNs are frequently used in NLP due to their ease of training and reputation as the best in sequence labelling. They depict the interdependence of all conceivable word combinations. Given the difficulties in identifying named entities for South African languages, including Setswana, and the inadequacy of resources, this research proposes the use of CNN model to identify named entities for Setswana. The results obtained are benchmarked with traditional methods such as Conditional random fields (CRF). The performance metrics such as F1-Score are explored in establishing the magnitude of trust and reliability of the proposed model. The model is evaluated using data from the South African Centre for Digital Language Resources' Setswana NER dataset. Compared to the present CRF model, which had an F-score performance of 78.0%, the testing results demonstrate that the model performs 94.0% better.

Multi-Meta Information Embedding Enhanced BERT for Chinese Mechanics Entity Recognition

The automatic extraction of key entities in mechanics problems is an important means to automatically solve mechanics problems. Nevertheless, for standard Chinese, compared with the open domain, mechanics problems have a large number of specialized terms and composite entities, which leads to a low recognition capability. Although recent research demonstrates that external information and pre-trained language models can improve the performance of Chinese Named Entity Recognition (CNER), few efforts have been made to combine the two to explore high-performance algorithms for extracting mechanics entities. Therefore, this article proposes a Multi-Meta Information Embedding Enhanced Bidirectional Encoder Representation from Transformers (MMIEE-BERT) for recognizing entities in mechanics problems. The proposed method integrates lexical information and radical information into BERT layers directly by employing an information adapter layer (IAL). Firstly, according to the characteristics of Chinese, a Multi-Meta Information Embedding (MMIE) including character embedding, lexical embedding, and radical embedding is proposed to enhance Chinese sentence representation. Secondly, an information adapter layer (IAL) is proposed to fuse the above three embeddings into the lower layers of the BERT. Thirdly, a Bidirectional Long Short-Term Memory (BiLSTM) network and a Conditional Random Field (CRF) model are applied to semantically encode the output of MMIEE-BERT and obtain each character’s label. Finally, extensive experiments were carried out on the dataset built by our team and widely used datasets. The results demonstrate that the proposed method has more advantages than the existing models in the entity recognition of mechanics problems, and the precision, recall, and F1 score were improved. The proposed method is expected to provide an automatic means for extracting key information from mechanics problems.

Evaluation of a prototype machine learning tool to semi-automate data extraction for systematic literature reviews

BackgroundEvidence-based medicine requires synthesis of research through rigorous and time-intensive systematic literature reviews (SLRs), with significant resource expenditure for data extraction from scientific publications. Machine learning may enable the timely completion of SLRs and reduce errors by automating data identification and extraction.MethodsWe evaluated the use of machine learning to extract data from publications related to SLRs in oncology (SLR 1) and Fabry disease (SLR 2). SLR 1 predominantly contained interventional studies and SLR 2 observational studies.Predefined key terms and data were manually annotated to train and test bidirectional encoder representations from transformers (BERT) and bidirectional long-short-term memory machine learning models. Using human annotation as a reference, we assessed the ability of the models to identify biomedical terms of interest (entities) and their relations. We also pretrained BERT on a corpus of 100,000 open access clinical publications and/or enhanced context-dependent entity classification with a conditional random field (CRF) model.Performance was measured using the F1 score, a metric that combines precision and recall. We defined successful matches as partial overlap of entities of the same type.ResultsFor entity recognition, the pretrained BERT+CRF model had the best performance, with an F1 score of 73% in SLR 1 and 70% in SLR 2. Entity types identified with the highest accuracy were metrics for progression-free survival (SLR 1, F1 score 88%) or for patient age (SLR 2, F1 score 82%). Treatment arm dosage was identified less successfully (F1 scores 60% [SLR 1] and 49% [SLR 2]). The best-performing model for relation extraction, pretrained BERT relation classification, exhibited F1 scores higher than 90% in cases with at least 80 relation examples for a pair of related entity types.ConclusionsThe performance of BERT is enhanced by pretraining with biomedical literature and by combining with a CRF model. With refinement, machine learning may assist with manual data extraction for SLRs.

Identification of mobile genetic elements with geNomad

Identifying and characterizing mobile genetic elements in sequencing data is essential for understanding their diversity, ecology, biotechnological applications and impact on public health. Here we introduce geNomad, a classification and annotation framework that combines information from gene content and a deep neural network to identify sequences of plasmids and viruses. geNomad uses a dataset of more than 200,000 marker protein profiles to provide functional gene annotation and taxonomic assignment of viral genomes. Using a conditional random field model, geNomad also detects proviruses integrated into host genomes with high precision. In benchmarks, geNomad achieved high classification performance for diverse plasmids and viruses (Matthews correlation coefficient of 77.8% and 95.3%, respectively), substantially outperforming other tools. Leveraging geNomad’s speed and scalability, we processed over 2.7 trillion base pairs of sequencing data, leading to the discovery of millions of viruses and plasmids that are available through the IMG/VR and IMG/PR databases. geNomad is available at https://portal.nersc.gov/genomad.

A management knowledge graph approach for critical infrastructure protection: Ontology design, information extraction and relation prediction

Critical Infrastructures (CI) underpin the basic functioning of society and the economy. Proper governance of CI security management remains a crucial challenge. This study aims to construct a knowledge graph for modeling CI protection. While the previous research has focused on threat intelligence modeling and open knowledge bases, they miss considering the defense side. Accordingly, we propose a knowledge graph for critical infrastructure protection, CIPKG, that extends the management ontology to include the defense side. It addresses the cross-industry and cross-time information gaps that occur in the process of CI protection management, making it more comprehensive in structure than the existing knowledge graph. We employ simplified Structured Threat Information Expression as attack ontology and design a new ontology for the defense side, which could combine with the existing threat ontology to form the CI protection knowledge graph. To dynamically extract information from emerging knowledge, we employ a Bi-directional Long Short-Term Memory and Conditional Random Field model with pre-trained cybersecurity domain-specific Bidirectional Encoder Representations from Transformers to recognize the named entities from CI regulations and standards. To associate the threat part with the management portion of the knowledge graph, we adopt the Knowledge Graph Bidirectional Encoder Representations from Transformer model to capture the semantic information and predict the relationship between threat and management. After information extraction and relation prediction, we build a knowledge graph with 529,360 nodes and about 3,335,000 edges.

Probabilistic multi-modal depth estimation based on camera–LiDAR sensor fusion

Multi-modal depth estimation is one of the key challenges for endowing autonomous machines with robust robotic perception capabilities. There have been outstanding advances in the development of uni-modal depth estimation techniques based on either monocular cameras, because of their rich resolution, or LiDAR sensors, due to the precise geometric data they provide. However, each of these suffers from some inherent drawbacks, such as high sensitivity to changes in illumination conditions in the case of cameras and limited resolution for the LiDARs. Sensor fusion can be used to combine the merits and compensate for the downsides of these two kinds of sensors. Nevertheless, current fusion methods work at a high level. They process the sensor data streams independently and combine the high-level estimates obtained for each sensor. In this paper, we tackle the problem at a low level, fusing the raw sensor streams, thus obtaining depth estimates which are both dense and precise, and can be used as a unified multi-modal data source for higher-level estimation problems. This work proposes a conditional random field model with multiple geometry and appearance potentials. It seamlessly represents the problem of estimating dense depth maps from camera and LiDAR data. The model can be optimized efficiently using the conjugate gradient squared algorithm. The proposed method was evaluated and compared with the state of the art using the commonly used KITTI benchmark dataset.

Research on Chinese Semantic Named Entity Recognition in Marine Engine Room Systems Based on BERT

With the development of intelligentization in maritime vessels, the pursuit of an organized and scalable knowledge storage approach for marine engine room systems has become one of the current research hotspots. This study addressed the foundational named entity recognition (NER) task in constructing a knowledge graph for marine engine rooms. It proposed an entity recognition algorithm for Chinese semantics in marine engine rooms that integrates language models. Firstly, the bidirectional encoder representation from transformers (BERT) language model is used to extract text features and obtain word-level granularity vector matrices. Secondly, the trained word embeddings are fed into a bidirectional long short-term memory network (BiLSTM) to extract contextual information. It considers the surrounding words and their sequential relationships, enabling a better understanding of the context. Additionally, the conditional random field (CRF) model was used to extract the globally optimal sequence of named entities in the marine engine room semantic. The CRF model considered the dependencies between adjacent entities that ensured a coherent and consistent final result for entity recognition in marine engine room semantics. The experiment results demonstrate that the proposed algorithm achieves superior F1 scores for all three entity types. Compared with BERT, the overall precision, recall, and F1 score of the entity recognition are improved by 1.36%, 1.41%, and 1.38%, respectively. Future research will be carried out on named entity recognition of a small sample set to provide basic support for more efficient entity relationship extraction and construction of a marine engine room knowledge graph.

Improving part-of-speech tagging in Amharic language using deep neural network

To date, several POS taggers have been introduced to facilitate the success of semantic analysis for different languages. However, the task of POS tagging becomes a bit intricate in morphologically complex languages, like Amharic. In this paper, we evaluated different models such as bidirectional long short term memory, convolutional neural network in combination with bidirectional long short term memory, and conditional random field for Amharic POS tagging. Various features, both language-dependent and -independent, have been explored in a conditional random field model. Besides, word-level and character-level features are analyzed in deep neural network models. A convolutional neural network is utilized for encoding features at the word and character level. Each model's performance has evaluated on the dataset that contained 321 K tokens and manually tagged with 31 POS tags. Lastly, the best performance obtained by an end-to-end deep neural network model, convolutional neural network in combination with bidirectional long term short memory and conditional random field, is 97.23% accuracy. This is the highest accuracy for Amharic POS tagging task and is competent with contemporary taggers currently existing in different languages.

Part-of-Speech Tagging of Odia Language Using Statistical and Deep Learning Based Approaches

Automatic part-of-speech (POS) tagging is a preprocessing step of many natural language processing tasks, such as named entity recognition, speech processing, information extraction, word sense disambiguation, and machine translation. It has already gained promising results in English and European languages. However, in Indian languages, particularly in the Odia language, it is not yet well explored because of the lack of supporting tools, resources, and morphological richness of the language. Unfortunately, we were unable to locate an open source POS tagger for the Odia language, and only a handful of attempts have been made to develop POS taggers for the Odia language. The main contribution of this research work is to present statistical approaches such as the maximum entropy Markov model and conditional random field (CRF), as well as deep learning based approaches, including the convolutional neural network (CNN) and bidirectional long short-term memory (Bi-LSTM) to develop the Odia POS tagger. A publicly accessible corpus annotated with the Bureau of Indian Standards (BIS) tagset is used in our work. However, most of the languages around the globe have used the dataset annotated with the Universal Dependencies (UD) tagset. Hence, to maintain uniformity, the Odia dataset should use the same tagset. Thus, following the BIS and UD guidelines, we constructed a mapping from the BIS tagset to the UD tagset. The maximum entropy Markov model, CRF, Bi-LSTM, and CNN models are trained using the Indian Languages Corpora Initiative corpus with the BIS and UD tagsets. We have experimented with various feature sets as input to the statistical models to prepare a baseline system and observed the impact of constructed feature sets. The deep learning based model includes the Bi-LSTM network, the CNN network, the CRF layer, character sequence information, and a pre-trained word vector. Seven different combinations of neural sequence labeling models are implemented, and their performance measures are investigated. It has been observed that the Bi-LSTM model with the character sequence feature and pre-trained word vector achieved a result with 94.58% accuracy.

Color design of modern architectural interior space based on environmental psychology

Abstract Due to the diversified development trend of modern architectural design, the functional design and application of houses are being presented in the interior space color layout of houses with its many advantages of interactivity, comprehensiveness, multi-functionality, and personalization. With this as the starting point, this paper analyzes the psychological perception of survey respondents on the same spatial color through big data, converts the perception into EEG signal for decoding, and after the decoding process of EEG data pre-processing, feature extraction, feature identification, and classification, calculates the significance of each element in the time-frequency matrix, which can get a homo-dimensional mask matrix. Then the conditional random field model is established on the random field theory to get the parameters of the model. Finally, the parameters of the model are obtained by maximizing the following entropy function, which is brought into the Lagrangian operator to obtain the pairwise Lagrangian operator. Finally, the EEG signal is decoded to realize the self-control training of color perception under different mentalities. The experimental results showed that by performing the intervention test on self-control and color insight, the mean value of the total self-control score in training was 61.99±11.45, and the intervention effect had stability. Therefore, improving self-control ability and forming correct tendency psychological perception plays a vital role in the color design of architectural space.

The self-control training method of vocal performance teaching in a new media environment

Abstract Due to the diverse development trend of modern media, new media arts and applications are being presented in the field of vocal performance teaching with its many advantages of interactivity, immediacy, sharing, comprehensiveness, versatility, community, and personalization. In this paper, by decoding the EEG signal, through the decoding process of EEG data pre-processing, feature extraction, feature identification, and classification, and calculating the significance of each element in the time-frequency matrix, an iso-dimensional mask matrix can be obtained. Then the conditional random field model is established on the random field theory to get the parameters of the model. Finally, the parameters of the model are obtained by maximizing the following entropy function, which is brought into the Lagrangian operator to obtain the pairwise Lagrangian operator. Finally, the EEG signal is decoded to realize the self-control training of vocal performance teaching in the new media environment. The experimental results show that by conducting the intervention test on self-control and vocal performance insight, the mean value of the total self-control score in self-control training is 61.99±11.45, and the intervention effect has stability. Therefore, improving self-control, forming correct expressions and forms, and enriching emotions are important for vocal performance.

Improved Node and Arc Multiplicity Estimation in De Bruijn Graphs Using Approximate Inference in Conditional Random Fields.

In de novo genome assembly using short Illumina reads, the accurate determination of node and arc multiplicities in a de Bruijn graph has a large impact on the quality and contiguity of the assembly. The multiplicity estimates of nodes and arcs guide the cleaning of the de Bruijn graph by identifying spurious nodes and arcs that correspond to sequencing errors. Additionally, they can be used to guide repeat resolution. Here, we model the entire de Bruijn graph and the accompanying read coverage information with a single Conditional Random Field (CRF) model. We show that approximate inference using Loopy Belief Propagation (LBP) on our model improves multiplicity assignment accuracy within feasible runtimes. The order in which messages are passed has a large influence on the speed of LBP convergence. Little theoretical guarantees exist and the conditions for convergence are not easily checked as our CRF model contains higher-order interactions. Therefore, we also present an empirical evaluation of several message passing schemes that may guide future users of LBP on CRFs with higher-order interactions in their choice of message passing scheme.

Chinese Event Extraction Method Based on Roformer Model

Event extraction is an important research direction in the field of natural language processing. The current Chinese event extraction field still suffers from errors in the pretraining and fine-tuning stages, inability to directly handle texts with more than 512 tokens, and inaccurate event extraction due to insufficient semantic sample diversity. In this paper, we propose a Chinese event extraction method RoformerFC (Roformer model with FGM and CRF) based on the Roformer model to address the above problems. Firstly, our method utilizes the Roformer model based on rotary position embedding, which both moderates the errors in the pretraining and fine-tuning phases and allows the model to directly handle texts with more than 512 tokens; then, the adversarial networks based on FGM (fast gradient method) are realized to increase the diversity of semantic feature samples; finally, the classical CRF (conditional random fields) model is used to decode and identify the event element entity and its corresponding event role and event type. On the short text DuEE dataset, the microP, microR, and microF of our method improved 1.26%, 4.01%, and 2.68%, respectively, over the classical Chinese event extraction method BERT-CRF. On the long text JsEE dataset, the microP, microR, and microF of our method improved 2.26%, 5.03%, and 3.72%, respectively, over the classical Chinese event extraction method BERT-CRF.

Automatic sentence segmentation for classical Chinese: The Spring and Autumn Annals as an example

Abstract There exists no sentence boundary in most classical Chinese literature texts. Since it is difficult to read literature of this kind, experts in literature or linguistics would segment the sentence manually. This article explores the effectiveness of classical Chinese sentence segmentation method so as to provide a reference for classical Chinese punctuation. On the basis of the machine learning methods, we chose three components of machine learning, namely models, tagging schemes, and features, to compare the learning results. The models include conditional random field (CRF) models, long short term memory (LSTM) models, BiLSTM–CRF models, and three Bidirectional Encoder Representation from Transformers (BERT) models. There are five tagging schemes in this article and three features including the statistical feature, Guangyun, and Fanqie. Finally, the performance of the combined feature template is evaluated by ten-fold cross-validation on four classical Chinese texts in different genres. The SikuBERT model is proved to be the most effective model for sentence segmentation at present. Different tagging schemes and various features are introduced. The results show that 5-tag-J tagging schemes can improve performance. Statistical feature, as an important clue for classical Chinese sentence segmentation, is useful in related tasks, but Guangyun and Fanqie have little impact. Other important factors of sentence segmentation are genres and writing styles.

Mine local homogeneous representation by interaction information clustering with unsupervised learning in histopathology images

Background and objective: The success of data-driven deep learning for histopathology images often depends on high-quality training sets and fine-grained annotations. However, as tumors are heterogeneous and annotations are expensive, unsupervised learning approaches are desirable to obtain full automation.Methods: In this paper, an Interaction Information Clustering (IIC) method is proposed to extract locally homogeneous features in mutually exclusive clusters. Trained in an unsupervised paradigm, the framework learns invariant information from multiple spatially adjacent regions for improved classification. Additionally, an adaptive Conditional Random Field (CRF) model is incorporated to detect spatially adjacent image patches of high morphological homogeneity in an offset-constraint free manner.Results: Empirically, the proposed model achieves an observable improvement of 11.4% on the downstream patch-level classification accuracy, compared with state-of-the-art unsupervised learning approaches.Conclusion: Furthermore, evaluated with our clinically collected histopathology whole-slide images, the proposed model shows high consistency in tissue distribution compared with well-trained supervised learning, which is of important diagnostic significance in clinical practice.

Design of a Structured Parsing Model for Corporate Bidding Documents Based on Bi-LSTM and Conditional Random Field (CRF)

International projects are often realized through the bidding process, but the existence of information exchange barriers in different countries leads to a complex and tedious bidding process. In this paper, the authors study the complex problem of reading bidding documents, combine the artificial intelligence to analyze them in a structured way, and realize the intelligent analysis of bidding documents. Firstly, through the CRF model, the structured analysis of the tender document is carried out according to the title of the tender document, and the extraction of quoted price, technology and commercial part is realized. Secondly, the detailed analysis of quoted price and technology is completed using the Bi-LSTM method, and the main five key feature extraction and analysis are completed. Finally, based on the CRF-Bi-LSTM method, the actual test is carried out, and the correlation coefficient is as high as 0.965. The results show that the structured parsing model proposed has good application prospects.

The Use of Knowledge Correlation for Classification and Evaluation of a Distance Education Teaching Resource Database


 Teaching information resource database is an important part of distance education in schools. Understanding the classification and characteristics of teaching resources in teaching information resource database can help teachers to search teaching resources more pertinently and make better use of these resources in actual teaching. Therefore, this article studies the classification and evaluation of distance education teaching resource database based on knowledge network. Combining the LSTM model and the conditional random field model, the resource entities that the target resource design knowledge points refer to are extracted from the resource text string of the teaching resource database. The resource entities that the extracted knowledge point refer to are matched and correlated with other resource entities related to the knowledge point in the teaching resource database, so as to realize the link of resource entities in the teaching resource database based on the concept of knowledge points. Starting with the feature extraction of resource text, this article innovates the resource classification algorithm of teaching resource database, and shows the new algorithm flow of feature extraction of resource text. Experimental results verify the effectiveness of the model.