Chain Conditional Random Field Research Articles

Literary Dialogue Across Cultures in the Context of Multiple Data Integration: New Essays on Comparative Literature and Poetics between China and the West

Abstract This study uses a two-stream multivariate data embedding network and FLAT model to effectively integrate text structure and linguistic information to enhance the performance of named entity recognition. In addition, named entity recognition and literary narrative analysis were carried out using linear chain conditional random field and Apriori algorithm, and literary character narratives and knowledge graphs were constructed. Finally, the characteristics and differences between Chinese and Western literature are deeply explored through econometric analysis and character similarity analysis, and it is found that the differences in geographic environment and socio-economic characteristics profoundly impact Chinese and Western literary art and poetics. The most fundamental difference between Chinese and Western societies lies in the commercial character of Western socio-economics, while China is more rural. The study reveals the uniqueness and intersection of the two civilizations in literary creation, providing a new perspective and theoretical support for cross-cultural academic dialogue.

Linear Chain Conditional Random Field for Operating Mode Identification and Multimode Process Monitoring.

As a supervised machine learning algorithm, conditional random fields are mainly used for fault classification, which cannot detect new unknown faults. In addition, faulty variable location based on them has not been studied. In this paper, conditional random fields with a linear chain structure are utilized for modeling multimode processes with transitions. A linear chain conditional random field model is trained by normal data with mode label. This model is able to distinguish transitions from stable modes well. After mode identification, the expectation of state feature function is developed for fault detection and faulty variable location. Case studies on the Tennessee Eastman process and continuous stirred tank reactor (CSTR) testify the effectiveness of the proposed approach.

Optimizing Small BERTs Trained for German NER

Currently, the most widespread neural network architecture for training language models is the so-called BERT, which led to improvements in various Natural Language Processing (NLP) tasks. In general, the larger the number of parameters in a BERT model, the better the results obtained in these NLP tasks. Unfortunately, the memory consumption and the training duration drastically increases with the size of these models. In this article, we investigate various training techniques of smaller BERT models: We combine different methods from other BERT variants, such as ALBERT, RoBERTa, and relative positional encoding. In addition, we propose two new fine-tuning modifications leading to better performance: Class-Start-End tagging and a modified form of Linear Chain Conditional Random Fields. Furthermore, we introduce Whole-Word Attention, which reduces BERTs memory usage and leads to a small increase in performance compared to classical Multi-Head-Attention. We evaluate these techniques on five public German Named Entity Recognition (NER) tasks, of which two are introduced by this article.

End-to-end masked graph-based CRF for joint slot filling and intent detection

Slot filling and intent detection are the basic and crucial fields of natural language processing (NLP) for understanding and analyzing human language, owing to their wide applications in real-world scenarios. Most existing methods of slot filling and intent detection tasks utilize linear chain conditional random field (CRF) for only optimizing slot filling, no matter the method is a pipeline or a joint model. In order to describe and exploit the implicit connections which indicate the appearance compatibility of different tag pairs, we introduce a graph-based CRF for a joint optimization of tag distribution of the slots and the intents. Instead of applying the complex inference algorithm of traditional graph-based CRF, we use an end-to-end method to implement the inference, which is formulated as a specialized multi-layer graph convolutional network (GCN). Furthermore, mask mechanism is introduced to our model for addressing multi-task problems with different tag-sets. Experimental results show the superiority of our model compared with other alternative methods. Our code is available athttps://github.com/tomsonsgs/e2e-mask-graph-crf.

Improved Crop Classification with Rotation Knowledge using Sentinel-1 and -2 Time Series

Leveraging the recent availability of accurate, frequent, and multimodal (radar and optical) Sentinel-1 and -2 acquisitions, this paper investigates the automation of land parcel identi- fication system (LPIS ) crop type classification. Our approach allows for the automatic integration of temporal knowledge, i.e., crop rotations using existing parcel-based land cover databases and multi-modal Sentinel-1 and -2 time series. The temporal evolution of crop types was modeled with a linear- chain conditional random field, trained with time series of multi-modal (radar and optical) satellite acquisitions and associated LPIS. Our model was tested on two study areas in France (≥ 1250 km2) which show different crop types, various parcel sizes, and agricultural practices: . the Seine et Marne and the Alpes de Haute-Provence classified accordingly to a fine national 25-class nomenclature. We first trained a Random Forest classifier without temporal structure to achieve 89.0% overall accuracy in Seine et Marne (10 classes) and 73% in Alpes de Haute-Provence (14 classes). We then demonstrated experimentally that taking into account the temporal structure of crop rotation with our model resulted in an increase of 3% to +5% in accuracy. This increase was especially important (+12%) for classes which were poorly classified without using the temporal structure. A stark posi- tive impact was also demonstrated on permanent crops, while it was fairly limited or even detrimental for annual crops.

Neural ParsCit: a deep learning-based reference string parser

We present a deep learning approach for the core digital libraries task of parsing bibliographic reference strings. We deploy the state-of-the-art long short-term memory (LSTM) neural network architecture, a variant of a recurrent neural network to capture long-range dependencies in reference strings. We explore word embeddings and character-based word embeddings as an alternative to handcrafted features. We incrementally experiment with features, architectural configurations, and the diversity of the dataset. Our final model is an LSTM-based architecture, which layers a linear chain conditional random field (CRF) over the LSTM output. In extensive experiments in both English in-domain (computer science) and out-of-domain (humanities) test cases, as well as multilingual data, our results show a significant gain ( $$p<0.01$$ ) over the reported state-of-the-art CRF-only-based parser.

CPS‐Rater: Automated Sequential Annotation for Conversations in Collaborative Problem‐Solving Activities

AbstractConversations in collaborative problem‐solving activities can be used to probe the collaboration skills of the team members. Annotating the conversations into different collaboration skills by human raters is laborious and time consuming. In this report, we report our work on developing an automated annotation system, CPS‐rater, for conversational data from collaborative activities. The linear chain conditional random field method is used to model the sequential dependencies between the turns of the conversations, and the resulting automated annotation system outperforms those systems that do not model the sequential dependency.

Improving prediction of burial state of residues by exploiting correlation among residues

BackgroundResidues in a protein might be buried inside or exposed to the solvent surrounding the protein. The buried residues usually form hydrophobic cores to maintain the structural integrity of proteins while the exposed residues are tightly related to protein functions. Thus, the accurate prediction of solvent accessibility of residues will greatly facilitate our understanding of both structure and functionalities of proteins. Most of the state-of-the-art prediction approaches consider the burial state of each residue independently, thus neglecting the correlations among residues.ResultsIn this study, we present a high-order conditional random field model that considers burial states of all residues in a protein simultaneously. Our approach exploits not only the correlation among adjacent residues but also the correlation among long-range residues. Experimental results showed that by exploiting the correlation among residues, our approach outperformed the state-of-the-art approaches in prediction accuracy. In-depth case studies also showed that by using the high-order statistical model, the errors committed by the bidirectional recurrent neural network and chain conditional random field models were successfully corrected.ConclusionsOur methods enable the accurate prediction of residue burial states, which should greatly facilitate protein structure prediction and evaluation.

The contribution of co-reference resolution to supervised relation detection between bacteria and biotopes entities.

BackgroundThe acquisition of knowledge about relations between bacteria and their locations (habitats and geographical locations) in short texts about bacteria, as defined in the BioNLP-ST 2013 Bacteria Biotope task, depends on the detection of co-reference links between mentions of entities of each of these three types. To our knowledge, no participant in this task has investigated this aspect of the situation. The present work specifically addresses issues raised by this situation: (i) how to detect these co-reference links and associated co-reference chains; (ii) how to use them to prepare positive and negative examples to train a supervised system for the detection of relations between entity mentions; (iii) what context around which entity mentions contributes to relation detection when co-reference chains are provided.ResultsWe present experiments and results obtained both with gold entity mentions (task 2 of BioNLP-ST 2013) and with automatically detected entity mentions (end-to-end system, in task 3 of BioNLP-ST 2013). Our supervised mention detection system uses a linear chain Conditional Random Fields classifier, and our relation detection system relies on a Logistic Regression (aka Maximum Entropy) classifier. They use a set of morphological, morphosyntactic and semantic features. To minimize false inferences, co-reference resolution applies a set of heuristic rules designed to optimize precision. They take into account the types of the detected entity mentions, and take advantage of the didactic nature of the texts of the corpus, where a large proportion of bacteria naming is fairly explicit (although natural referring expressions such as "the bacteria" are common). The resulting system achieved a 0.495 F-measure on the official test set when taking as input the gold entity mentions, and a 0.351 F-measure when taking as input entity mentions predicted by our CRF system, both of which are above the best BioNLP-ST 2013 participant system.ConclusionsWe show that co-reference resolution substantially improves over a baseline system which does not use co-reference information: about 3.5 F-measure points on the test corpus for the end-to-end system (5.5 points on the development corpus) and 7 F-measure points on both development and test corpora when gold mentions are used. While this outperforms the best published system on the BioNLP-ST 2013 Bacteria Biotope dataset, we consider that it provides mostly a stronger baseline from which more work can be started. We also emphasize the importance and difficulty of designing a comprehensive gold standard co-reference annotation, which we explain is a key point to further progress on the task.

Machine Reading for Extraction of Bacteria and Habitat Taxonomies.

There is a vast amount of scientific literature available from various resources such as the internet. Automating the extraction of knowledge from these resources is very helpful for biologists to easily access this information. This paper presents a system to extract the bacteria and their habitats, as well as the relations between them. We investigate to what extent current techniques are suited for this task and test a variety of models in this regard. We detect entities in a biological text and map the habitats into a given taxonomy. Our model uses a linear chain Conditional Random Field (CRF). For the prediction of relations between the entities, a model based on logistic regression is built. Designing a system upon these techniques, we explore several improvements for both the generation and selection of good candidates. One contribution to this lies in the extended exibility of our ontology mapper that uses an advanced boundary detection and assigns the taxonomy elements to the detected habitats. Furthermore, we discover value in the combination of several distinct candidate generation rules. Using these techniques, we show results that are significantly improving upon the state of art for the BioNLP Bacteria Biotopes task.

Beyond linear chain

Information Extraction (IE) is a field at the crossroads of IR and NLP that studies methods for extracting information from text in such a way that this information can be used to populate a structured information repository. The main methods by means of which IE has been tackled rely on supervised learning;the best-performing such methods belong to the class of probabilistic graphical models, and, in particular, to the class of Conditional Random Fields (CRFs). In this thesis we investigate two major aspects related to textual IE via CRFs: (a) the creation of CRFs models that can outperform the commonly adopted linear- chain CRFs, and the creation of methods for ensuring the quality of training data and for assessing the impact of training data quality on the accuracy of CRFs systems for IE. We start by facing the task of IE from medical documents written in the Italian language. We propose two novel approaches: (i) a cascaded, two-stage method composed by two layers of CRFs, and (ii) a confidence-weighted ensemble method that combines standard linear-chain CRFs and the proposed two-stage method. Both the proposed models are shown to outperform a standard linear-chain CRFs system. We then investigate aspect-oriented sentence-level opinion mining from product reviews, that consists in predicting, for all sentences in the review, whether the sentence expresses a positive, neutral, or negative opinion (or no opinion at all) about a specific aspect of the product. We propose a set of increasingly powerful models based on CRFs, including a hierarchical multi-label CRFs scheme that jointly models the overall opinion expressed in a product review and the set of aspect-specific opinions expressed in each of its sentences. The proposed CRFs models are shown to obtain better results than linear-chain CRFs. We then study the impact that the quality of training data has on the accuracy of an IE system via experiments performed on a dataset in which inter-coder agreement data are available. Finally, we investigate active learning techniques for a type of semi-supervised CRFs specifically devised for partially labeled sequences. We show that margin-based strategies always obtain the best results on the four tasks we have tested them on.

A Knowledge Engineering Framework for Identifying Key Impact Factors from Safety‐Related Accident Cases

Consumer product safety closely relates to consumer health. In this paper, a knowledge engineering framework is proposed for data mining to identify key safety factors from a large number of consumer product safety cases. Data mining in the framework is performed in three steps. The first step is to collect consumer product safety cases, a case can be semistructured or unstructured, and cases can be collected either manually or automatically by a web spider crawling certain websites. The second step is to extract all safety factors from a number of consumer product safety cases. A new method based on linear chain conditional random field is developed to extract safety factors. The effectiveness of the method has been validated on product cases. The third step is to identify a set of key factors from all safety factors by knowledge reasoning. To illustrate the process of knowledge reasoning, a set of 3192 safety cases of electric products with electric shock accidents is chosen as the case study; a Bayesian network based model is developed to retrieve key safety factors relating to electric shock accidents. The performance of the reasoning model has been verified by a combination of experts' evaluation and experiments, and it has shown the proposed reasoning model can help identify key safety factors of electric shock accidents successfully. Overall, the proposed framework is capable of identifying key safety factors from a large number of consumer product safety cases. Copyright © 2014 John Wiley &amp; Sons, Ltd.

Tool wear state recognition based on linear chain conditional random field model

Tool condition monitoring (TCM) system is paramount for guaranteeing the quality of workpiece and improving the efficiency of the machining process. To overcome the shortcomings of Hidden Markov Model (HMM) and improve the accuracy of tool wear recognition, a linear chain conditional random field (CRF) model is presented. As a global conditional probability model, the main characteristic of this method is that the estimation of the model parameters depends not only on the current feature vectors but also on the context information in the training data. Therefore, it can depict the interrelationship between the feature vectors and the tool wear states accurately. To test the effectiveness of the proposed method, acoustic emission data are collected under four kinds of tool wear state and seven statistical features are selected to realize the tool wear classification by using CRF and hidden Markov model (HMM) based pattern recognition method respectively. Moreover, k-fold cross validation method is utilized to estimate the generation error accurately. The analysis and comparison under different folds schemes show that the CRF model is more accurate for the classification of the tool wear state. Moreover, the stability and the training speed of the CRF classifier outperform the HMM model. This method casts some new lights on the tool wear monitoring especially in the real industrial environment.

A comparative study of pattern recognition classifiers to predict physical activities using smartphones and wearable body sensors

This paper presents a wireless body area network platform that performs physical activities recognition using accelerometers, biosignals and smartphones. Multiple classifiers and sensor combinations were examined to identify the classifier with the best recognition performance for the static and dynamic activities. The Functional Trees classifier proved to provide the best results among the classifiers evaluated (Naive Bayes, Bayesian Networks, Support Vector Machines and Decision Trees [C4.5, Random Forest]) and was used to train the model which was implemented for the real time activity recognition on the smartphone. The identified patterns of daily physical activities were used to examine conformance with medical advice, regarding physical activity guidelines. An algorithm based on Skip Chain Conditional Random Fields, received as inputs the recognized activities and data retrieved from the GPS receiver of the smartphone to develop dynamic daily patterns that enhance prediction results. The presented platform can be extended to be used in the prevention of short-term complications of metabolic diseases such as diabetes.