Error Propagation Problem Research Articles

GFN: A novel joint entity and relation extraction model with redundancy and denoising strategies

Joint entity and relation extraction refers to the extraction of entities and their corresponding relationships in the given sentence, which has gained increasing attention in recent years. Some joint extraction models utilize a shared encoder to model the interactions between named entity recognition and relation extraction subtasks. Despite achieving decent performance, they inevitably face the issue of error propagation. One-step exhaustive methods can mitigate the error propagation problem to some extent, but they suffer from issues such as huge computation complexity and a proliferation of negative samples. Therefore, addressing the problems mentioned above, we propose a Greedy Filter Network that combines Greedy-NER and Filter-RE. GFN employs the Greedy-NER with a redundancy strategy to prioritize recall, thereby reducing error propagation between subtasks. To reduce the computational complexity, we design an innovative approach to represent and store spans in Greedy-NER. In Filter-RE, we traverse all pairwise combinations of candidate entities. To address the issue of widespread negative samples, we design a denoising strategy with two filters, effectively filtering out entity pairs without relations, which can eliminate noise and alleviate the issue of negative sample proliferation. Finally, to enable flexible control over the redundancy strategy, we design two misclassifying penalty parameters for each module. The experimental results indicate that GFN achieves the state-of-the-art F1-score on the CoNLL04 and NYT datasets, with a notable 2.0% improvement observed specifically on CoNLL04.

Neural models for semantic analysis of handwritten document images

Semantic analysis of handwritten document images offers a wide range of practical application scenarios. A sequential combination of handwritten text recognition (HTR) and a task-specific natural language processing system offers an intuitive solution in this domain. However, this HTR-based approach suffers from the problem of error propagation. An HTR-free model, which avoids explicit text recognition and solves the task end-to-end, tackles this problem, but often produces poor results. A possible reason for this is that it does not incorporate largely pre-trained semantic word embeddings, which turn out to be one of the most powerful advantages in the textual domain. In this work, we propose an HTR-based and an HTR-free model and compare them on a variety of segmentation-based handwritten document image benchmarks including semantic word spotting, named entity recognition, and question answering. Furthermore, we propose a cross-modal knowledge distillation approach to integrate semantic knowledge from textually pre-trained word embeddings into HTR-free models. In a series of experiments, we investigate optimization strategies for robust semantic word image representation. We show that the incorporation of semantic knowledge is beneficial for HTR-free approaches in achieving state-of-the-art results on a variety of benchmarks.

Nested Entity Recognition Method Based on Multidimensional Features and Fuzzy Localization

Nested named entity recognition (NNER) aims to identify potentially overlapping named entities. Sequence labeling method and span-based method are two commonly used methods in nested named entity recognition. However, the linear structure of sequence labeling method results in relatively poor performance, and span-based method requires traversing all spans, which brings very high time complexity. All of them fail to effectively leverage the positional dependencies between internal and external entities. In order to improve these issues, this paper proposed a nested entity recognition method based on Multidimensional Features and Fuzzy Localization (MFFL). Firstly, this method adopted the shared encoding that fused three features of characters, words, and parts of speech to obtain a multidimensional feature vector representation of the text and obtained rich semantic information in the text. Secondly, we proposed to use the fuzzy localization to assist the model in pinpointing the potential locations of entities. Finally, in the entity classification, it used a window to expand the sub-sequence and enumerate possible candidate entities and predicted the classification labels of these candidate entities. In order to alleviate the problem of error propagation and effectively learn the correlation between fuzzy localization and classification labels, we adopted multi-task learning strategy. This paper conducted several experiments on two public datasets. The experimental results showed that the proposed method achieves ideal results in both nested entity recognition and non-nested entity recognition tasks, and significantly reduced the time complexity of nested entity recognition.

Joint signal-to-signal beat interference mitigation for the field recovery of symmetric carrier-assisted differential detection with low carrier-to-signal power ratio

As a combination of direct detection and coherent detection technologies, self-coherent detection has the advantages of low cost and optical field recovery ability. However, most of the self-coherent detection techniques are limited to single sideband (SSB) signals. Recently, carrier-assisted differential detection (CADD) has been proposed to realize complex-valued double sideband (DSB) signals, but it requires a high carrier-to-signal power ratio (CSPR) to mitigate the signal-to-signal beat interference (SSBI). Later, a more cost-effective symmetric CADD (S-CADD) has been proposed while the required CSPR is still high. In order to alleviate the high requirements of CSPR, we propose a scheme based on the joint of digital pre-distortion (DPD) at transmitter and clipping at receiver to further improve the S-CADD system performance. This joint processing can not only solve the problem of non-uniform distribution of subcarrier signal-to-noise ratio (SNR) caused by non-ideal transfer function, but also the error propagation problem caused by enhanced SSBI under low CSPR. After the validation of the 64 Gbaud 16-ary quadrature amplitude modulation (16-QAM) orthogonal frequency division multiplexing (OFDM) signal transmitted over 80 km standard single mode fiber (SSMF), the CSPR required by the proposed scheme to reach the 20% soft decision-forward error correction (SD-FEC) and 7% hard decision-forward error correction (HD-FEC) can be reduced by 1.3 dB and 2.8 dB, respectively, with a comparison of the conventional S-CADD. The results show the potential of the proposed scheme in the short-reach optical transmissions.

A syntactic features and interactive learning model for aspect-based sentiment analysis

The aspect-based sentiment analysis (ABSA) consists of two subtasks: aspect term extraction (AE) and aspect term sentiment classification (ASC). Previous research on the AE task has not adequately leveraged syntactic information and has overlooked the issue of multi-word aspect terms in text. Current researchers tend to focus on one of the two subtasks, neglecting the connection between the AE and ASC tasks. Moreover, the problem of error propagation easily occurs between two independent subtasks when performing the complete ABSA task. To address these issues, we present a unified ABSA model based on syntactic features and interactive learning. The proposed model is called syntactic interactive learning based aspect term sentiment classification model (SIASC). To overcome the problem of extracting multi-word aspect terms, the model utilizes part-of-speech features, words features, and dependency features as textual information. Meanwhile, we designs a unified ABSA structure based on the end-to-end framework, reducing the impact of error propagation issues. Interaction learning in the model can establish a connection between the AE task and the ASC task. The information from interactive learning contributes to improving the model’s performance on the ASC task. We conducted an extensive array of experiments on the Laptop14, Restaurant14, and Twitter datasets. The experimental results show that the SIASC model achieved average accuracy of 84.11%, 86.65%, and 78.42% on the AE task, respectively. Acquiring average accuracy of 81.35%, 86.71% and 76.56% on the ASC task, respectively. The SIASC model demonstrates superior performance compared to the baseline model.

Recursive label attention network for nested named entity recognition

Nested named entity recognition (NER) has been mainly studied by recognizing inner entities first, then outer entities, or vice versa, depending on the nesting level. However, the previous models on nested NER do not deal with the data sparseness problems of deep nested entities (i.e., insufficiency of training data for deeper nested entities) and the error propagation problems between nested entities (i.e., accumulation of errors caused by inner entities or outer entities). To alleviate the data sparseness problems, we propose a recursive label attention network. In contrast to previous models, the proposed model explicitly reflects the nested level and effectively re-uses lower level label information through level-reflected label embeddings. To counteract error propagation in the recursive architecture, we propose an inner entity pretraining strategy in which the proposed model is sequentially trained from lower to higher levels. In experiments, we use task-specific metrics for nested NER, unlike the evaluation metrics used in previous studies. The experimental results show that the proposed model is particularly effective in cases where entities of the same type were nested. In addition, it showed high F1-scores even when named entities are deeply nested. This fact reveals that our proposed model is effective for nest NER.

Synthetical reasoning for hybrid tabular-textual question answering over multitasking model

Hybrid tabular-textual question answering (QA) is a crucial task in natural language processing that involves reasoning and locating answers from various information sources, primarily through numerical reasoning and span extraction. Cur-rent techniques in numerical reasoning often rely on autoregressive models to decode program sequences. However, these methods suffer from exposure bias and error propagation, which can significantly decrease the accuracy of program generation as the decoding process unfolds. To address these challenges, this paper proposes a novel multitasking hybrid tabular-textual question answering (MHTTQA) framework. Instead of generating operators and operands step by step, this framework can independently generate entire program tuples in parallel. This innovative approach solves the problem of error propagation and greatly improves the speed of program generation. The effectiveness of the method is demonstrated through experiments using the ConvFinQA and MultiHiertt datasets. Our proposed model outperforms the strong FinQANet baselines by 7% and 7.2% Exe/Prog Acc and the MT2Net baselines by 20.9% and 9.4% EM/F1. In addition, the program generation rate of our method far exceeds that of the baseline method. Additionally, our non-autoregressive program generation method exhibits greater resilience to an increasing number of numerical reasoning steps, further highlighting the advantages of our proposed framework in the field of hybrid tabular-textual QA.

An ensemble multi-scale framework for long-term forecasting of air quality.

The significance of accurate long-term forecasting of air quality for a long-term policy decision for controlling air pollution and for evaluating its impacts on human health has attracted greater attention recently. This paper proposes an ensemble multi-scale framework to refine the previous version with ensemble empirical mode decomposition (EMD) and nonstationary oscillation resampling (NSOR) for long-term forecasting. Within the proposed ensemble multi-scale framework, we on one hand apply modified EMD to produce more regular and stable EMD components, allowing the long-range oscillation characteristics of the original time series to be better captured. On the other hand, we provide an ensemble mechanism to alleviate the error propagation problem in forecasts caused by iterative implementation of NSOR at all lead times and name it improved NSOR. Application of the proposed multi-scale framework to long-term forecasting of the daily PM2.5 at 14 monitoring stations in Hong Kong demonstrates that it can effectively capture the long-term variation in air pollution processes and significantly increase the forecasting performance. Specifically, the framework can, respectively, reduce the average root-mean-square error and the mean absolute error over all 14 stations by 8.4% and 9.2% for a lead time of 100 days, compared to previous studies. Additionally, better robustness can be obtained by the proposed ensemble framework for 180-day and 365-day long-term forecasting scenarios. It should be emphasized that the proposed ensemble multi-scale framework is a feasible framework, which is applicable for long-term time series forecasting in general.

Joint extraction of biomedical overlapping triples through feature partition encoding

Entities and relations extraction are the key tasks in the construction of biomedical knowledge graph, which play an important role in the biomedical artificial intelligence. However, extraction of entities and relations from biomedical texts is challenging because of the overlapping triples problem. The previous approaches typically divided the task into two separate sub-tasks. However, these methods failed to address the error propagation problem. Recent methods have been proposed to perform both sub-tasks simultaneously. Nonetheless, most current methods still encounter issues related to imbalanced interactions and independent features. In this paper, we propose a novel method based on feature partition encoding and relative positional embedding to joint extract biomedical entity and relation triples simultaneously. Compared to previous work, our method shows exceptional accurate in extracting entities and relations, while efficiently tackling the challenge of overlapping triples in biomedical texts. Our work has two contributions. Firstly, our method divides the features into task-specific and shared parts through entity, relation and sharing partitions at the encoding stage. And the encoded features will be aggregated according to the subsequent tasks. Secondly, we introduce a relative positional embedding method to capture the relative distance information between token pairs. In this way, our method can effectively deal with the sub-tasks interactions problem and improve entities and relations extraction. The experimental results show that our method improves the F1 scores of relations extraction by 3.2%, 2.1%, 3.4%, and 2.8% on four biomedical datasets, respectively.

Unequal Error Protection-based Rate-Splitting precoding for VLC systems: BER analysis

Rate-Splitting Multiple Access (RSMA), as a promising next-generation multiple access technique, can be integrated with Visible Light Communication (VLC) to address the challenge of accommodating multiple users under limited modulation bandwidth of Light Emitting Diodes (LEDs). However, most of the current works about RSMA assume perfect Successive Interference Cancellation (SIC) during the decoding procedure and neglect the error propagation problem due to SIC errors, which will greatly affect the performance in real applications. In view of this, we consider the design of Rate-Splitting (RS) precoder for downlink Multi-User Multiple-Input Single-Output (MU-MISO) VLC systems in the presence of SIC errors. For the first time in the literature, we derive analytic Bit Error Rate (BER) expressions in this case and analyze the error performance of a conventional RS precoder, which unveils the error performance degradation due to SIC errors. To mitigate this effect and improve the robustness of the system, we propose a novel RS precoding scheme by providing Unequal Error Protection (UEP) for common and private streams, which is achieved by changing an error protection factor. By this means, the transmission power is flexibly allocated to the common and private symbol stream, which can reduce the error probability as a whole. Simulation results indicate that the proposed RS precoder can provide significant error performance gains (up to 82.9%) over the baseline.

An effective relation-first detection model for relational triple extraction

Relational triple extraction is a crucial task in the field of information extraction, which attempts to identify all triples from natural language text. Existing methods primarily focus on addressing the issue of overlapping triples. However, the majority of studies need to perform the same operation on all predefined relations when solving this problem, which will lead to relation redundancy. In addition, most methods have the problem of error propagation. During training, they use the ground truth labels as a priori knowledge to predict at different stages, while during inference, they must use the labels predicted in the previous stage to predict in the following stages. To address these problems, we propose an effective relation-first detection model for relational triple extraction (ERFD-RTE). The proposed model first detects the potential relations in the sentence and then performs entity recognition for each specific relation, which aims to solve the overlapping triples issue and avoid additional calculations for redundant relations. We design a random label error strategy for the error propagation problem in the training phase, which balances the difference between training and inference. Experiment results demonstrate that ERFD-RTE is superior to other baselines by improving the F1 score to 92.7% (+0.7%) on NYT-P and NYT-E, 92.9% (+0.3%) on WebNLG-P, 89.3% (+0.9%) on WebNLG-E and 83.71% (+1.5%) on ADE. Additional analysis shows that ERFD-RTE can effectively extract overlapping triples.

A two-step log-linear procedure for graphical representation and inference of associations in cross-classified data for disease diagnosis.

Biometrical sciences and disease diagnosis in particular, are often concerned with the analysis of associations for cross-classified data, for which distance association models give us a graphical interpretation for non-sparse matrices with a low number of categories. In this framework, usually binary exploratory and response variables are present, with analysis based on individual profiles being of great interest. For saturated models, we show the usual linear relationship for log-linear models is preserved in full dimension for the distance association parameterization. This enables a two-step procedure to facilitate the analysis and the interpretation of associations in terms of unfolding after the overall and main effects are removed. The proposed procedure can deal with cross-classified data for profiles by binary variables, and it is easy to implement using traditional statistical software. For disease diagnosis, the problems of a degenerate solution in the unfolding representation, and that of determining significant differences between the profile locations are addressed. A hypothesis test of independence based on odds ratio is considered. Furthermore, a procedure is proposed to determine the causes of the significance of the test, avoiding the problem of error propagation. The equivalence between a test for equality of odds ratio pairs and the test for equality of location for two profiles in the unfolding representation in the disease diagnosis is shown. The results have been applied to a real example on the diagnosis of coronary disease, relating the odds ratios with performance parameters of the diagnostic test.

SPBERE: Boosting span-based pipeline biomedical entity and relation extraction via entity information

Triplet extraction is one of the fundamental tasks in biomedical text mining. Compared with traditional pipeline approaches, joint methods can alleviate the error propagation problem from entity recognition to relation classification. However, existing methods face challenges in detecting overlapping entities and overlapping relations, which are ubiquitous in biomedical texts. In this work, we propose a novel pipeline method of end-to-end biomedical triplet extraction. In particular, a span-based detection strategy is used to detect the overlapping triplets by enumerating possible candidate spans and entity pairs. The strategy is further used to capture different contextualized representations via an entity model and a relation model, respectively. Furthermore, to enhance interrelation between spans, entity information from the output of the entity model is used to construct the input for the relation model without utilizing any external knowledge. Our approach is evaluated on the drug–drug interaction (DDI) and chemical-protein interaction (CHEMPROT) datasets, exhibiting improvement of the absolute F1-score in relation extraction by 3.5%–3.7% compared prior work. The experimental results highlight the importance of overlapping triplet detection using the span-based approach, acquisition of various contextualized representations via different in-domain pre-trained language models, and early fusion of entity information in the relation model.

A Scope Sensitive and Result Attentive Model for Multi-Intent Spoken Language Understanding

Multi-Intent Spoken Language Understanding (SLU), a novel and more complex scenario of SLU, is attracting increasing attention. Unlike traditional SLU, each intent in this scenario has its specific scope. Semantic information outside the scope even hinders the prediction, which tremendously increases the difficulty of intent detection. More seriously, guiding slot filling with these inaccurate intent labels suffers error propagation problems, resulting in unsatisfied overall performance. To solve these challenges, in this paper, we propose a novel Scope-Sensitive Result Attention Network (SSRAN) based on Transformer, which contains a Scope Recognizer (SR) and a Result Attention Network (RAN). SR assignments scope information to each token, reducing the distraction of out-of-scope tokens. RAN effectively utilizes the bidirectional interaction between SF and ID results, mitigating the error propagation problem. Experiments on two public datasets indicate that our model significantly improves SLU performance (5.4% and 2.1% on Overall accuracy) over the state-of-the-art baseline.

MIGA: A Unified Multi-Task Generation Framework for Conversational Text-to-SQL

Conversational text-to-SQL is designed to translate multi-turn natural language questions into their corresponding SQL queries. Most advanced conversational text-to-SQL methods are incompatible with generative pre-trained language models (PLMs), such as T5. In this paper, we present a two-stage unified MultI-task Generation frAmework (MIGA) that leverages PLMs’ ability to tackle conversational text-to-SQL. In the pre-training stage, MIGA first decomposes the main task into several related sub-tasks and then unifies them into the same sequence-to-sequence (Seq2Seq) paradigm with task-specific natural language prompts to boost the main task from multi-task training. Later in the fine-tuning stage, we propose four SQL perturbations to alleviate the error propagation problem. MIGA tends to achieve state-of-the-art performance on two benchmarks (SparC and CoSQL). We also provide extensive analyses and discussions to shed light on some new perspectives for conversational text-to-SQL.

Iterative pseudo-labelling with SoftMax probability in text classification

Semi-supervised learning is one of the potential research fields in text classification. In this paper, semi-supervised pseudo-label training experiments are conducted using the BERT model that has been pre-trained as a baseline. Only 20% of the original dataset is used for the new training set after segmenting the training set. The raw corpus used for pseudo-label training consists of the remaining 80% of data after labels are removed, while the original test set is still utilized. The results indicate that the key to the semi-supervised pseudo-labelling method is the performance of the original model and reasonable data filtering techniques. Even though the SoftMax value used for data filtering is not precisely equivalent to model prediction accuracy, experimental results show it can somewhat reduce the error propagation problem of the model. This is consistent with earlier research. However, using SoftMax as the threshold for data screening can't bring enough benefits to the model training and make it surpass the training performance of the original data set. As a result, future studies will focus on improving the accuracy of pseudo-labelling with a more suitable data selection method to better the model's performance.

Uplink and Downlink NOMA Based on a Novel Interference Coefficient Estimation Strategy for Next-Generation Optical Wireless Networks

Non-orthogonal multiple access (NOMA) has been widely recognized as a promising technology to improve the transmission capacity of wireless optical communication systems. NOMA considers the principle of successive interference cancellation (SIC) to separate a user’s signal at the receiver side. To improve the ability of optical signal detection, we developed a quantum dot (QD) fluorescent concentrator incorporated with multiple-input and single-output (MISO) to realize an uplink NOMA-based optical wireless system. However, inaccurate interference assessment of multiple users using the SIC detection algorithm at the receiver side may lead to more prominent error propagation problems and affect the bit error rate (BER) performance of the system. This research aims to propose a novel recurrent neural network-based guided frequency interference coefficient estimation algorithm in a NOMA visible light communication (VLC) system. This algorithm can improve the accuracy of interference estimation compared with the traditional SIC detection algorithm by introducing interference coefficients. It provides a more accurate reconstruction possibility for level-by-level interference cancellation and weakens the influence of error propagation. In addition, we designed uplink and downlink NOMA-VLC communication systems for experimental validation. When the power allocation ratio was in the range of 0.8 to 0.97, the experimental results of the downlink validated that the BER performance of both users satisfied the forward error correction (FEC) limit with the least squares (LS)-SIC and the long short-term memory recurrent neural networks (LSTM)-SIC detection strategy. Moreover, the BER performance of the LSTM-SIC algorithm was better than that of the LS-SIC algorithm for all users when the power allocation ratio was in the range of 0.92 to 0.93. In particular, our proposed system offered a large detection area of 2 cm2 and corresponding aggregate data rate up to 40 Mbps over 1.5 m of free space by using QDs, and we successfully achieved a mean bit error rate (BER) of 2.3 × 10−3 for the two users.

MGRC: An End-to-End Multigranularity Reading Comprehension Model for Question Answering.

Deep neural network-based models have achieved great success in extractive question answering. Recently, many works have been proposed to model multistage matching for this task, which usually first retrieve relevant paragraphs or sentences and then extract an answer span from the retrieved results. However, such a pipeline-based approach suffers from the error propagation problem, especially for sentence-level retrieval that is usually difficult to achieve high accuracy due to the severe data imbalance problem. Furthermore, since the paragraph/sentence selector and the answer extractor are closely related, modeling them independently does not fully exploit the power of multistage matching. To solve these problems, we propose a novel end-to-end multigranularity reading comprehension model, which is a unified framework to explicitly model three matching granularities, including paragraph identification, sentence selection, and answer extraction. Our approach has two main advantages. First, the end-to-end approach alleviates the error propagation problem in both the training and inference phases. Second, the shared features in a unified model improve the learning of representations of different matching granularities. We conduct a comprehensive comparison on four large-scale datasets (SQuAD-open, NewsQA, SQuAD 2.0, and SQuAD Adversarial) and verify that the proposed approach outperforms both the vanilla BERT model and existing multistage matching approaches. We also conduct an ablation study and verify the effectiveness of the proposed components in our model structure.

BTDM: A Bi-Directional Translating Decoding Model-Based Relational Triple Extraction

The goal of relational triple extraction is to extract knowledge-rich relational triples from unstructured text. Although the previous methods obtain considerable performance, there are still some problems, such as error propagation, the overlapping triple problem, and suboptimal subject–object alignment. To address the shortcomings above, in this paper, we decompose this task into three subtasks from a fresh perspective: entity extraction, subject–object alignment and relation judgement, as well as propose a novel bi-directional translating decoding model (BTDM). Specifically, a bidirectional translating decoding structure is designed to perform entity extraction and subject–object alignment, which decodes entity pairs from both forward and backward extraction. The bidirectional structure effectively mitigates the error propagation problem and aligns the subject–object pairs. The translating decoding approach handles the overlapping triple problem. Finally, a (entity pair, relation) bipartite graph is designed to achieve practical relationship judgement. Experiments show that our model outperforms previous methods and achieves state-of-the-art performance on NYT and WebNLG. We achieved F1-scores of 92.7% and 93.8% on the two datasets. Meanwhile, in various complementary experiments on complex scenarios, our model demonstrates consistent performance gain in various complex scenarios.

A Span-based Target-aware Relation Model for Frame-semantic Parsing

Frame-semantic Parsing (FSP) is a challenging and critical task in Natural Language Processing (NLP). Most of the existing studies decompose the FSP task into frame identification (FI) and frame semantic role labeling (FSRL) subtasks, and adopt a pipeline model architecture that clearly causes error propagation problem. However, recent jointly learning models aim to address the above problem and generally treat FSP as a span-level structured prediction task, which, unfortunately, leads to cascading error propagation problem between roles and less-efficient solutions due to huge search space of roles. To address these problems, we reformulate the FSRL task into a target-aware relation classification task and propose a novel and lightweight jointly learning framework that simultaneously processes three subtasks of FSP, including frame identification, argument identification, and role classification. The novel task formulation and jointly learning with interaction mechanisms among subtasks can help improve the overall system performance and reduce the search space and time complexity, compared with existing methods. Extensive experimental results demonstrate that our proposed model significantly outperforms 10 state-of-the-art models in terms of F1 score across two benchmark datasets.