Error Detection Task Research Articles

Heterogeneous Views and Spatial Structure Enhancement for triple error detection

Knowledge graph error detection is to identify erroneous triples in knowledge graphs that are inconsistent with objective facts in the real world. In practice, the quality of knowledge graphs is an indispensable foundation for the widespread and accurate knowledge application services such as intelligent retrieval and human machine dialogue. Technically, the existing knowledge graph error detection methods face the following two problems: few available negative samples of triples, and an uneven data distribution. That distribution is caused by the large disparity in the number of head and tail entities belonging to the same relationship and the disparity in the number of triples with different relationships. To alleviate these problems, this paper proposes an approach based on the Heterogeneous Views and Spatial Structure Enhancement (HVSSE) in a contrastive learning framework for triple error detection task. Specifically, the heterogeneous views are constructed to include four kinds of triple views, i.e., positive and negative triple views based on head or tail entity co-occurrence. Moreover, Graph-Spatial-Transformer with an explicit spatial structure encoding is designed to fully capture the contextual information of triple nodes. Thereby, driven by the framework of contrastive learning, our HVSSE model can not only learn more discriminative embedding of triples, but also capture the local structure of triples and global contextual information of knowledge graphs. Experimental results on five public datasets indicate that our proposed approach is superior to the state-of-the-art methods, showing its the effectiveness on triple error detection.

Reading Comprehension and Comprehension Monitoring in First-grade Children with Vocabulary Delay

Objectives: This study investigated reading comprehension and reading comprehension monitoring abilities of children with vocabulary delay at the end of the first grade. Specifically, the study examined whether children with vocabulary delays could detect internal inconsistency errors, which are contradictory pieces of information in a short story, and whether they could correct those errors to make appropriate changes to the story at the beginning stage of reading. Methods: Fourteen children with vocabulary delay and 15 typically developing children participated in this study. Each child read 10 short stories without internal inconsistency errors and 10 stories with internal inconsistency errors. After reading the text, children were asked to find any inconsistent parts of the story and correct them. Results: The results showed that children with vocabulary delay performed worse on reading comprehension than their peers. They also scored statistically lower on both the error detection and error correction tasks in stories with internal inconsistency errors compared to typically developing children. For typically developing children only, there were significant correlations between reading comprehension and reading comprehension monitoring. Conclusion: Children with vocabulary delay exhibited difficulties in both reading comprehension and comprehension monitoring. Poor vocabulary knowledge can hinder their ability to recognize when a text is misunderstood or to apply effective strategies for resolving problems, ultimately leading to reading comprehension failure. This suggests that children with language disorders, including those with vocabulary delays, need to be assessed and provided with interventions for both reading comprehension and comprehension monitoring.

Single-step parity check gate set for quantum error correction

A key requirement for an effective quantum error correction (QEC) scheme is that the physical qubits have error rates below a certain threshold. The value of this threshold depends on the details of the specific QEC scheme, and its hardware-level implementation. This is especially important with parity-check circuits, which are the fundamental building blocks of QEC codes. The standard way of constructing the parity check circuit is using a universal set of gates, namely sequential CNOT gates, single-qubit rotations and measurements. We exploit the insight that a QEC code does not require universal logic gates, but can be simplified to perform the sole task of error detection and correction. By building gates that are fundamental to QEC, we can boost the threshold and ease the experimental demands on the physical hardware. We present a rigorous formalism for constructing and verifying the error behavior of these gates, linking the physical measurement of a process matrix to the abstract error models commonly used in QEC analysis. This allows experimentalists to directly map the gates used in their systems to thresholds derived for a broad-class of QEC codes. We give an example of these new constructions using the model system of two nuclear spins, coupled to an electron spin, showing the potential benefits of redesigning fundamental gate sets using QEC primitives, rather than traditional gate sets reliant on simple single and two-qubit gates.

Beyond extraction accuracy: addressing the quality of geographical named entity through advanced recognition and correction models using a modified BERT framework

ABSTRACT In the realm of geospatial services and applications, the accuracy of address information is of utmost importance. Traditional methods of data collection, being both labor-intensive and costly, have prompted researchers to turn to Volunteered Geographic Information (VGI) for the extraction of Geographical Named Entity (GNE).Notwithstanding, prior studies have predominantly concentrated on enhancing extraction accuracy, while often overlooking the critical aspect of GNE quality. This study addresses this gap by employing a multifaceted approach. Initially, a Geographical Named Entity Semantic Model (GNESM) was constructed by improving the BERT framework and conducting ablation experiments on multiple influencing factors to verify its feasibility. Based on GNESM, a Geographical Named Entity Recognition Model (GNERM) was constructed by incremental pre-training with social media text data and fine-tuning to achieve a recognition accuracy of 90.9%. Subsequently, a Geographical Named Entity Error Correction Model (GNEECM) was constructed by training GNESM with standard GNE data and incorporating error detection and correction modules, achieving a remarkable accuracy of 96.6% in error detection and correction tasks. The experimental results convincingly demonstrate that the proposed identification and correction methods outperform all compared methods. Through the identification and correction process, this study successfully obtained high-quality GNE data, providing a reference for expanding standard address libraries and subsequent research on geographic named entity.

Relation of life sciences students’ metacognitive monitoring to neural activity during biology error detection

Metacognitive calibration—the capacity to accurately self-assess one’s performance—forms the basis for error detection and self-monitoring and is a potential catalyst for conceptual change. Limited brain imaging research on authentic learning tasks implicates the lateral prefrontal and anterior cingulate brain regions in expert scientific reasoning. This study aimed to determine how variation in undergraduate life sciences students’ metacognitive calibration relates to their brain activity when evaluating the accuracy of biological models. Fifty undergraduate students enrolled in an introductory life sciences course completed a biology model error detection task during fMRI. Students with higher metacognitive calibration recruited lateral prefrontal regions linked in prior research to expert STEM reasoning to a greater extent than those with lower metacognitive calibration. Findings suggest that metacognition relates to important individual differences in undergraduate students’ use of neural resources during an authentic educational task and underscore the importance of fostering metacognitive calibration in the classroom.

Humans Supervising Artificial Intelligence – Investigation of Designs to Optimize Error Detection

ABSTRACT Artificial Intelligence (AI) fundamentally changes the way we work by introducing new capabilities. Human tasks shift towards a supervising role where the human confirms or disconfirms the presented decision. In this study, we utilise the signal detection theory to investigate and explain how the performance of human error detection is influenced by specific information design. We conducted two online experiments in the context of AI-supported information extraction and measured the ability of participants to validate the extracted information. In the first experiment, we investigated the mechanism of information provided prior to conducting the error detection task. In the second experiment, we manipulated the design of the presented information during the task and investigated its effect. Both manipulations significantly impacted the error detection performance of humans. Hence our study provides important insights for developing AI-based decision support systems and contributes to the theoretical understanding of human-AI collaboration.

Triple trustworthiness evaluation for knowledge graph of industrial domain

The knowledge graph is widely used in industrial fields due to its structural characteristics. In order to reduce the cost of wrong decision-making, it is more important for the industrial knowledge graph to guarantee the quality and comprehensiveness of knowledge. In order to obtain the trustworthiness information of triples in the graph, this paper proposed a model to evaluate triple trustworthiness for industrial knowledge graphs(TT-IKG) and obtains the final score of triples by fusing the output of three sub-modules that investigate the local confidence of triples, the confidence of schema-matching and the confidence of global paths. Experiments on the real industrial knowledge graph of enterprises verified the effectiveness of the model, and the experimental results on the error detection and graph completion tasks of the knowledge graph show that the effect is better than that of other models.

The Challenges of Teaching and Assessing Technical Translation in an Era of Neural Machine Translation

Teaching translation in higher education has undeniably been impacted by the innovations brought about by machine translation (MT), more particularly neural machine translation (NMT). This influence has become significantly more noticeable in recent years, as NMT technology progresses hand in hand with artificial intelligence. A case study supported by a questionnaire conducted among translation students (bachelor’s and master’s programmes at ISCAP) probed the degree of student satisfaction with CAT tools and revealed that they favour the use of MT in their translation practices, focusing their work on post-editing tasks rather than exploring other translation strategies and complementary resources. Although MT cannot be disregarded in translation programmes, as machine-generated translations make up an increasingly larger amount of a professional translator’s output, the widespread use of MT by students poses new challenges to translators’ training, since it becomes more difficult to assess students’ level of proficiency. Translation teachers must not only adapt their classroom strategies to accommodate these current translation strategies (NMT) but also, as intended by this study, find new, adequate methods of training and assessing students that go beyond regular translation assignments while still ensuring that students acquire the proper translation competence. Thus, as the use of NMT makes it considerably more challenging to assess a student’s level of translation competence, it is necessary to introduce other activities that not only allow students to acquire and develop their translation competence as defined in the EMT (European Masters in Translation) framework but also enable teachers to assess students more objectively. Hence, this article foregrounds a set of activities usually regarded as “indirect tasks” for technical translation courses that hopefully results in the development of student translation skills and competence, as well as provides more insights for teachers on how to more objectively assess students. It is possible, then, to conclude that these activities, such as different types of paraphrasing and error-detection tasks, may have the potential to encourage creative thinking and problem-solving strategies, giving teachers more resources to assess students’ level of translation competence.

Annotation Error Detection: Analyzing the Past and Present for a More Coherent Future

AbstractAnnotated data is an essential ingredient in natural language processing for training and evaluating machine learning models. It is therefore very desirable for the annotations to be of high quality. Recent work, however, has shown that several popular datasets contain a surprising number of annotation errors or inconsistencies. To alleviate this issue, many methods for annotation error detection have been devised over the years. While researchers show that their approaches work well on their newly introduced datasets, they rarely compare their methods to previous work or on the same datasets. This raises strong concerns on methods’ general performance and makes it difficult to assess their strengths and weaknesses. We therefore reimplement 18 methods for detecting potential annotation errors and evaluate them on 9 English datasets for text classification as well as token and span labeling. In addition, we define a uniform evaluation setup including a new formalization of the annotation error detection task, evaluation protocol, and general best practices. To facilitate future research and reproducibility, we release our datasets and implementations in an easy-to-use and open source software package.1

Don't overthink it: The paradoxical nature of expertise for the detection of errors in conceptual business process models.

Business process models are widely used artifacts in design activities to facilitate communication about business domains and processes. Despite being an extensively researched topic, some aspects of conceptual business modeling are yet to be fully explored and understood by academicians and practitioners alike. We study the attentional characteristics specific to experts and novices in a semantic and syntactic error detection task across 75 Business Process Model and Notation (BPMN) models. We find several intriguing results. Experts correctly identify more error-free models than novices, but also tend to find more false positive defects. Syntactic errors are diagnosed faster than semantic errors by both groups. Both groups spend more time on error-free models. Our findings regarding the ambiguous differences between experts and novices highlight the paradoxical nature of expertise and the need to further study how best to train business analysts to design and evaluate conceptual models.

An Automatic Error Detection Method for Engineering English Translation Based on the Deep Learning Model

Accuracy of deep learning model translation is a key index to evaluate the application performance of engineering English translation. In this paper, an automatic error detection system for English translation is proposed. In the particular task of grammar detection, researchers have gradually shifted their attention from statistical methods to neural network methods. Three deep learning algorithm models are established, and the multitask performance of the model is better than that of the conditional random field model and the LSTM-CRF model. The reason is that the multitask learning model of auxiliary tasks is included to some extent, which solves the problem of data sparsity and enables the model to be fully trained even under the condition of uneven label distribution. Thus, it performs better than other models in the task of syntax error detection. It realizes the word spelling error check based on the dictionary and uses the thought of editing distance to prompt the word error found, which can automatically check a large number of translations. On the basis of analyzing the sentence structure characteristics of engineering English translation, this paper realizes the detection of subject-verb agreement errors and analyzes the main word of the subject corresponding to the predicate verb by constructing the syntactic structure tree of the sentence, so as to realize the judgment of subject-verb agreement errors.

A Stacked Generalization Chest-X-Ray-Based Framework for Mispositioned Medical Tubes and Catheters Detection

Tubes and catheters are medical devices introduced into the human body to help ill patients in critical health conditions. However, several positioning errors occur during or after the placement of such devices (Endotracheal tubes mispositioned in 10 to 20% of intubations). In addition, the delay of X-ray diagnosis after surgery can cause serious complications. Such delays are caused by the hospitals' resourcelessness or due to workload in intensive care units. The X-rays images availability (Most used diagnosis modality in intensive care units, 40% to 50%) and the presence of tubes in those images (lines are present on 33% of X-ray images) present a fertile ground to feed DCNNs training on tube error detection tasks and reduce complications. However, training and tuning one DCNN learner to resolve tube detection is time-consuming. Therefore, we propose a custom stacked generalization framework to combine wake learners with a proposed meta learner neural network architecture to resolve tube error detection tasks. The proposed framework AUC (93.84%) outperforms other related work methods with the input size of (380pixel*380pixel). Furthermore, we demonstrated the sensibility of stacked generalization to the number of base learners. Moreover, we validated the utility of input cross-validation used to form level1-metadata for the stacked generalization. Our framework can be adapted to be integrated with a CAD (computer aid decision system) for tubes error detection. The CAD can detect errors immediately after patient screening and notify radiologists to prioritize diagnosis of cases with positioning errors to adjust tubes and reduce risks significantly.

Construction of an Artificial Intelligence Writing Model for English Based on Fusion Neural Network Model.

This paper presents an in-depth study and analysis of the model of English writing using artificial intelligence algorithms of neural networks. Based on word vectors, the unsupervised disambiguation, and clustering of multimedia contexts extracted from massive online videos, the disambiguation accuracy reaches over 0.7, and the resulting small-scale multimedia context set can cover up to 90% of vocabulary learning tasks; user experiments show that the multimedia context learning system based on this method can improve the effectiveness and experience of ESL vocabulary learning, as well as the long-term word sense memory of learners. The results are 30% better. Based on the dependency grammatical relations and semantic metrics of collocations on a large-scale professional corpus, we established a collocation intention description and retrieval method in line with users' linguistic cognition and doubled the usage rate of collocation retrieval on the actual deployment system after half a year, becoming a user “sticky” ESL writing aid, and further defined style. Dictionaries only provide basic lexical definitions, and, even if supported by example sentences, they still cannot meet the needs of ESL authors in terms of expressive accuracy and richness. However, the current machine translation is based on the black box deep neural network construction, and its translation process is not understandable and interactive. Among the three algorithmic models constructed in this paper, the multitask learning model outperforms the conditional random field model and the LSTM-CRF model because the multitask learning model with auxiliary tasks solves the problem of sparse data to a certain extent, allowing the model to be trained more adequately in the case of uneven label distribution, and thus performs better than other models in the task of grammatical error detection.

Effects of Score Study and Conducting Gesture on Collegiate Musicians’ Ability to Detect Errors in a Choral Score

The purpose of this study was to examine the effects of score study and conducting gesture on collegiate musicians’ ability to detect errors in a choral score. Is there a combined effect of gesture and score study that impacts undergraduate conductors’ identification of errors in a score? Participants ( N = 53) viewed a sequence of four choral score excerpts presented via Zoom video conferencing. We asked participants to identify errors under one of four conditions: score study with a correct model recording, conducting with a simple timekeeping pattern while listening, both score study and conducting, or neither. After listening to each excerpt, participants notified the researcher about the exact location, voice part, and error type of any error that they heard. There were significant differences among conditions, with post hoc tests indicating superior error detection scores for the score study conditions. Conducting during the error detection task resulted in lower error detection scores, especially when not preceded by score study with a correct aural model.

Music-experience-related and musical-error-dependent activations in the brain.

Although music is one of human-unique traits such as language, its neural basis for cortical organization has not been well understood. In the present functional magnetic resonance imaging study, we tested an error-detection task with different types of musical error (pitch, tempo, stress, and articulation conditions) and examined three groups of secondary school students having different levels of music experience. First, we observed distinct activation patterns under these music conditions, such that specific activations under the pitch condition were consistently replicated for all tested groups in the auditory areas, as well as in the left language areas under the articulation condition. Second, music-experience-related activations were observed in multiple regions, including the right sensorimotor area under the pitch condition, as well as in the right premotor cortex under the articulation condition. Indeed, the right homologs of the language areas were specifically activated under the stress and articulation conditions. Third, activations specific to the group with the highest proficiency in music were observed under the tempo condition mostly in the right regions. These results demonstrate the existence of music-related signatures in the brain activations, including both universal and experience-related mechanisms.

Machine Learning-Based Grammar Error Detection Method in English Composition

The detection of grammatical errors in English composition is an important task in the field of NLP. The main purpose of this task is to check out grammatical errors in English sentences and correct them. Grammatical error detection and correction are important applications in the automatic proofreading of English texts and in the field of English learning aids. With the increasing influence of English on a global scale, a huge breakthrough has been made in the task of detecting English grammatical errors. Based on machine learning, this paper designs a new method for detecting grammatical errors in English composition. First, this paper implements a grammatical error detection model based on Seq2Seq. Second, this paper implements a grammatical error detection and correction scheme based on the Transformer model. The Transformer model performs better than most grammar models. Third, this paper realizes the application of the BERT model in grammar error detection and error correction tasks, and the generalization ability of the model has been significantly enhanced. This solves the problem that the forward and backward cannot be merged when the Transformer trains the language model. Fourth, this paper proposes a method of grammatical error detection and correction in English composition based on a hybrid model. According to specific application scenarios, the corresponding neural network model is used for grammatical error correction. Combine the Seq2Seq structure to encode the input sequence and automate feature engineering. Through the combination of traditional model and deep model, the advantages are complemented to realize grammatical error detection and automatic correction.

Analysis of Japanese Expressions and Semantics Based on Link Sequence Classification

Based on locness corpus, this paper uses Wordsmith 6.0, SPSS 24, and other software to explore the use of temporal connectives in Japanese writing by Chinese Japanese learners. This paper proposes a method of tense classification based on the Japanese dependency structure. This method analyzes the results of the syntactic analysis of Japanese dependence and combines the tense characteristics of the target language to extract tense‐related information and construct a maximum entropy tense classification model. The model can effectively identify the tense, and its classification accuracy shows the effectiveness of the classification method. This paper proposes a temporal feature extraction algorithm oriented to the hierarchical phrase expression model. The end‐to‐end speech recognition system has become the development trend of large‐scale continuous speech recognition because of its simplicity and efficiency. In this paper, the end‐to‐end technology based on link timing classification is applied to Japanese speech recognition. Taking into account the characteristics of Japanese hiragana, katakana, and Japanese kanji writing forms, through experiments on the Japanese data set, different suggestions are explored. The final effect is better than mainstream speech recognition systems based on hidden Markov models and two‐way long and short‐term memory networks. This algorithm can extract the temporal characteristics of rules that meet certain conditions while extracting expression rules. These tense characteristics can guide the selection of rules in the expression process, make the expression results more in line with linguistic knowledge, and ensure the choice of relevant vocabulary and the structural ordering of the language. Through the analysis of time series and static information, we combine the time and space dimensions of the network structure. Using connectionist temporal classification (CTC) technology, an end‐to‐end speech recognition method for pronunciation error detection and diagnosis tasks is established. This method does not require phonemic information nor does it require forced alignment. The extended initials and finals are the error primitives, and 64 types of errors are designed. The experimental results show that the method can effectively detect the wrong pronunciation, the detection accuracy rate is 87.07%, the false rejection rate is 7.83%, and the error rate is 87.07%. The acceptance rate is 25.97%. This method uses network information more comprehensively than traditional methods, and the model is more effective. After detailed experiments, this article evaluates the prediction effect of this method and previous methods on the data set. This method improves the prediction accuracy by about 15% and achieves the expected goal of the work in this paper.

A Bayesian end-to-end model with estimated uncertainties for simple question answering over knowledge bases

Existing methods for question answering over knowledge bases (KBQA) ignore the consideration of the model prediction uncertainties. We argue that estimating such uncertainties is crucial for the reliability and interpretability of KBQA systems. Therefore, we propose a novel end-to-end KBQA model based on Bayesian Neural Network (BNN) to estimate uncertainties arose from both model and data. To our best knowledge, we are the first to consider the uncertainty estimation problem for the KBQA task using BNN. The proposed end-to-end model integrates entity detection and relation prediction into a unified framework, and employs BNN to model entity and relation under the given question semantics, transforming network weights into distributions. Therefore, predictive distributions can be estimated by sampling weights and forward inputs through the network multiple times. Uncertainties can be further quantified by calculating the variances of predictive distributions. The experimental results demonstrate the effectiveness of uncertainties in both the misclassification detection task and cause of error detection task. Furthermore, the proposed model also achieves comparable performance compared to the existing state-of-the-art approaches on SimpleQuestions dataset.

Implicit learning of two artificial grammars.

This study investigated the implicit learning of two artificial systems. Two finite-state grammars were implemented with the same tone set (leading to short melodies) and played by the same timbre in exposure and test phases. The grammars were presented in separate exposure phases, and potentially acquired knowledge was tested with two experimental tasks: a grammar categorization task (Experiment 1) and a grammatical error detection task (Experiment 2). Results showed that participants were able to categorize new items as belonging to one or the other grammar (Experiment 1) and detect grammatical errors in new sequences of each grammar (Experiment 2). Our findings suggest the capacity of intra-modal learning of regularities in the auditory modality and based on stimuli that share the same perceptual properties.

Error Detection for Arabic Text Using Neural Sequence Labeling

The English language has, thus far, received the most attention in research concerning automatic grammar error correction and detection. However, these tasks have been less investigated for other languages. In this paper, we present the first experiments using neural network models for the task of error detection for Modern Standard Arabic (MSA) text. We investigate several neural network architectures and report the evaluation results acquired by applying cross-validation on the data. All experiments involve a corpus we created and augmented. The corpus has 494 sentences and 620 sentences after augmentation. Our models achieved a maximum precision of 78.09%, recall of 83.95%, and F0.5 score of 79.62% in the error detection task using SimpleRNN. Using an LSTM, we achieved a maximum precision of 79.21%, recall of 93.8%, and F0.5 score of 79.16%. Finally, the best results were achieved using a BiLSTM with a maximum precision of 80.74%, recall of 85.73%, and F0.5 score of 81.55%. We compared the results of the three models to a baseline, which is a commercially available Arabic grammar checker (Microsoft Word 2007). LSTM, BiLSTM, and SimpleRNN all outperformed the baseline in precision and F0.5. Our work shows preliminary results, demonstrating that neural network architectures for error detection through sequence labeling can successfully be applied to Arabic text.