Reduce Error Propagation Research Articles

Entity and relation extractions for threat intelligence knowledge graphs

Advanced persistent threats (APTs) represent a complex challenge in cybersecurity as they infiltrate networks stealthily to conduct espionage, steal data, and maintain a long-term presence. To combat these threats, security professionals increasingly rely on cyber knowledge graphs (CKGs), which provide scalable solutions to analyze and structure vast amounts of cyber threat intelligence (CTI) from diverse sources in real-time, enabling the automation of proactive security measures. Developing CKGs requires extracting entity and their relationships from unstructured CTI reports. However, existing approaches face significant limitations, such as difficulties with the nuances of cybersecurity language, diverse threat terminologies, and high rates of error propagation, resulting in low accuracy and poor generalizability. This paper introduces a novel Threat Intelligence Knowledge Graph (TiKG) pipeline designed to address these challenges. The TiKG framework leverages SecureBERT, a domain-specific transformer-based model optimized for cybersecurity, and integrates it with an attention-based BiLSTM to capture the context and nuances of security texts, reducing error propagation and improving extraction accuracy. Additionally, the pipeline incorporates a domain-specific ontology and inference model to ensure precise relation mapping in relation extraction. Using three large-scale TI open-source datasets (DNRTI, STUCCO, and CYNER) and a curated CTI dataset, extensive evaluations demonstrate the effectiveness of our framework, showing significant improvements over existing methods in detecting and linking cyber threats. These contributions provide a robust platform for security professionals to analyze and predict potential attacks, develop effective defenses, and enhance the strategic capabilities of cybersecurity operations.

TGIN: Translation-Based Graph Inference Network for Few-Shot Relational Triplet Extraction.

Extracting relational triplets aims at detecting entity pairs and their semantic relations. Compared with pipeline models, joint models can reduce error propagation and achieve better performance. However, all of these models require large amounts of training data, therefore performing poorly on many long-tail relations in reality with insufficient data. In this article, we propose a novel end-to-end model, called TGIN, for few-shot triplet extraction. The core of TGIN is a multilayer heterogeneous graph with two types of nodes (entity node and relation node) and three types of edges (relation-entity edge, entity-entity edge, and relation-relation edge). On the one hand, this heterogeneous graph with entities and relations as nodes can intuitively extract relational triplets jointly, thereby reducing error propagation. On the other hand, it enables the triplet information of limited labeled data to interact better, thus maximizing the advantage of this information for few-shot triplet extraction. Moreover, we devise a graph aggregation and update method that utilizes translation algebraic operations to mine semantic features while retaining structure features between entities and relations, thereby improving the robustness of the TGIN in a few-shot setting. After updating the node and edge features through layers, TGIN propagates the label information from a few labeled examples to unlabeled examples, thus inferring triplets from these unlabeled examples. Extensive experiments on three reconstructed datasets demonstrate that TGIN can significantly improve the accuracy of triplet extraction by 2.34%~10.74% compared with the state-of-the-art baselines. To the best of our knowledge, we are the first to introduce a heterogeneous graph for few-shot relational triplet extraction.

Reduced-order modeling of unsteady fluid flow using neural network ensembles

The use of deep learning has become increasingly popular in reduced-order models (ROMs) to obtain low-dimensional representations of full-order models. Convolutional autoencoders (CAEs) are often used to this end as they are adept at handling data that are spatially distributed, including solutions to partial differential equations. When applied to unsteady physics problems, ROMs also require a model for time-series prediction of the low-dimensional latent variables. Long short-term memory (LSTM) networks, a type of recurrent neural network useful for modeling sequential data, are frequently employed in data-driven ROMs for autoregressive time-series prediction. When making predictions at unseen design points over long time horizons, error propagation is a frequently encountered issue, where errors made early on can compound over time and lead to large inaccuracies. In this work, we propose using bagging, a commonly used ensemble learning technique, to develop a fully data-driven ROM framework referred to as the CAE-eLSTM ROM that uses CAEs for spatial reconstruction of the full-order model and LSTM ensembles for time-series prediction. When applied to two unsteady fluid dynamics problems, our results show that the presented framework effectively reduces error propagation and leads to more accurate time-series prediction of latent variables at unseen points.

Thermal characterization of thin films: A chip-based approach for in-plane property analysis

Accurate measurement of thermal properties in thin films is crucial for optimizing devices and deepening our understanding of heat transfer at nano and micro scales. This study presents a combined experimental and computational investigation on a chip-integrated technique for the assessment of in-plane thermal properties of thin films. This method stands out by incorporating inherent error cancelation to lessen the impact of radiative heat loss and allows simultaneous, independent determination of both thermal conductivity and diffusivity through straightforward linear fittings from the same dataset, reducing error propagation. We examine an 84 nm thick SiNx membrane over a temperature range from 100 K to nearly 500 K, aligning with previous studies. Further investigations into a conducting polymer film post-doping demonstrate a notable increase in both thermal conductivity and diffusivity, corroborating scanning thermal microscopy observations, confirming the technique's efficacy and reliability.

GFN: A novel joint entity and relation extraction model with redundancy and denoising strategies

Joint entity and relation extraction refers to the extraction of entities and their corresponding relationships in the given sentence, which has gained increasing attention in recent years. Some joint extraction models utilize a shared encoder to model the interactions between named entity recognition and relation extraction subtasks. Despite achieving decent performance, they inevitably face the issue of error propagation. One-step exhaustive methods can mitigate the error propagation problem to some extent, but they suffer from issues such as huge computation complexity and a proliferation of negative samples. Therefore, addressing the problems mentioned above, we propose a Greedy Filter Network that combines Greedy-NER and Filter-RE. GFN employs the Greedy-NER with a redundancy strategy to prioritize recall, thereby reducing error propagation between subtasks. To reduce the computational complexity, we design an innovative approach to represent and store spans in Greedy-NER. In Filter-RE, we traverse all pairwise combinations of candidate entities. To address the issue of widespread negative samples, we design a denoising strategy with two filters, effectively filtering out entity pairs without relations, which can eliminate noise and alleviate the issue of negative sample proliferation. Finally, to enable flexible control over the redundancy strategy, we design two misclassifying penalty parameters for each module. The experimental results indicate that GFN achieves the state-of-the-art F1-score on the CoNLL04 and NYT datasets, with a notable 2.0% improvement observed specifically on CoNLL04.

TGIN: Document-level event extraction with two-phase graph inference network

Document-level event extraction aims to extract event records from a whole document that contain numerous entities scattered across multiple sentences. Efficiently modeling the interactions among these entities is crucial. However, previous methods suffer from two main shortcomings. Firstly, they tend to implicitly model key information, which can result in representations with higher levels of noise. Secondly, they excessively consider irrelevant entities, thereby reducing extraction efficiency and precision. To address these issues, we propose a novel Two-phase Graph Inference Network (TGIN) approach for extracting document-level events. In the first phase, TGIN constructs a heterogeneous document-level graph to capture complex interactions among nodes of different granularity, enabling the acquisition of document-aware features. Subsequently, a dedicated module is developed to extract relevant entity pairs within the same event record. This module utilizes a key information aggregator with an attention mechanism to explicitly aggregate key sentences for entity pairs. In the second phase, the entity links predicted in the first phase serve as prior information to construct the entity-level graph, which focuses on modeling interactions between entity pairs that potentially share the same event link, effectively reducing error propagation. Experimental results on the publicly available document-level event extraction dataset ChFinAnn demonstrate the superiority of our framework over most existing models.

AACFlow: an end-to-end model based on attention augmented convolutional neural network and flow-attention mechanism for identification of anticancer peptides.

Anticancer peptides (ACPs) have natural cationic properties and can act on the anionic cell membrane of cancer cells to kill cancer cells. Therefore, ACPs have become a potential anticancer drug with good research value and prospect. In this article, we propose AACFlow, an end-to-end model for identification of ACPs based on deep learning. End-to-end models have more room to automatically adjust according to the data, making the overall fit better and reducing error propagation. The combination of attention augmented convolutional neural network (AAConv) and multi-layer convolutional neural network (CNN) forms a deep representation learning module, which is used to obtain global and local information on the sequence. Based on the concept of flow network, multi-head flow-attention mechanism is introduced to mine the deep features of the sequence to improve the efficiency of the model. On the independent test dataset, the ACC, Sn, Sp, and AUC values of AACFlow are 83.9%, 83.0%, 84.8%, and 0.892, respectively, which are 4.9%, 1.5%, 8.0%, and 0.016 higher than those of the baseline model. The MCC value is 67.85%. In addition, we visualize the features extracted by each module to enhance the interpretability of the model. Various experiments show that our model is more competitive in predicting ACPs.

ID-SF-Fusion: a cooperative model of intent detection and slot filling for natural language understanding

PurposeIntent detection (ID) and slot filling (SF) are two important tasks in natural language understanding. ID is to identify the main intent of a paragraph of text. The goal of SF is to extract the information that is important to the intent from the input sentence. However, most of the existing methods use sentence-level intention recognition, which has the risk of error propagation, and the relationship between intention recognition and SF is not explicitly modeled. Aiming at this problem, this paper proposes a collaborative model of ID and SF for intelligent spoken language understanding called ID-SF-Fusion.Design/methodology/approachID-SF-Fusion uses Bidirectional Encoder Representation from Transformers (BERT) and Bidirectional Long Short-Term Memory (BiLSTM) to extract effective word embedding and context vectors containing the whole sentence information respectively. Fusion layer is used to provide intent–slot fusion information for SF task. In this way, the relationship between ID and SF task is fully explicitly modeled. This layer takes the result of ID and slot context vectors as input to obtain the fusion information which contains both ID result and slot information. Meanwhile, to further reduce error propagation, we use word-level ID for the ID-SF-Fusion model. Finally, two tasks of ID and SF are realized by joint optimization training.FindingsWe conducted experiments on two public datasets, Airline Travel Information Systems (ATIS) and Snips. The results show that the Intent ACC score and Slot F1 score of ID-SF-Fusion on ATIS and Snips are 98.0 per cent and 95.8 per cent, respectively, and the two indicators on Snips dataset are 98.6 per cent and 96.7 per cent, respectively. These models are superior to slot-gated, SF-ID NetWork, stack-Prop and other models. In addition, ablation experiments were performed to further analyze and discuss the proposed model.Originality/valueThis paper uses word-level intent recognition and introduces intent information into the SF process, which is a significant improvement on both data sets.

Optimized distance measures on hesitant fuzzy numbers: An application

As the rapidly progressing applications of uncertainty theories, the need for modifications to some of their existing mathematical tools or creating new tools to deal correctly with them in various environments is also exposed. Hesitant fuzzy numbers (HFNs), as a particular case of fuzzy numbers, are not an exception to this rule. Considering the necessity of determining the distance between given HFNs in many of their practical applications, this article shows that the existing methods either do not provide correct results or are not able to meet the needs of users. This paper aims to present new methods for distance measures of hesitant fuzzy numbers. To do them, three prevalent distance measures, i.e., the generalized distance measure, the Hamming distance measure, and the Euclidean distance measure, will be optimized into three distinct trinal categories. With the approach of reducing error propagation via reducing some unnecessary mathematical computations, new distance measures on HFNs will be introduced, first. The middle is the modification of the first category, which is more suitable when the given HFNs are equal-distance by the previous formula. Also, as the third category, the weighted form of these distance measures has been proposed, to be used where the real and membership parts of HFNs are not of equal importance. As an application of these, a TOPSIS-based technique for solving multi-attribute group decision-making problems with HFNs has been proposed. A numerical example will be implemented to describe the presented method. Finally, along with the validation of the proposed method, its numerical comparison with some other existing methods will be discussed in detail.

AutoSourceID-Classifier

Aims.Traditional star-galaxy classification techniques often rely on feature estimation from catalogs, a process susceptible to introducing inaccuracies, thereby potentially jeopardizing the classification’s reliability. Certain galaxies, especially those not manifesting as extended sources, can be misclassified when their shape parameters and flux solely drive the inference. We aim to create a robust and accurate classification network for identifying stars and galaxies directly from astronomical images.Methods.The AutoSourceID-Classifier (ASID-C) algorithm developed for this work uses 32x32 pixel single filter band source cutouts generated by the previously developed AutoSourceID-Light (ASID-L) code. By leveraging convolutional neural networks (CNN) and additional information about the source position within the full-field image, ASID-C aims to accurately classify all stars and galaxies within a survey. Subsequently, we employed a modified Platt scaling calibration for the output of the CNN, ensuring that the derived probabilities were effectively calibrated, delivering precise and reliable results.Results.We show that ASID-C, trained on MeerLICHT telescope images and using the Dark Energy Camera Legacy Survey (DECaLS) morphological classification, is a robust classifier and outperforms similar codes such as SourceExtractor. To facilitate a rigorous comparison, we also trained an eXtreme Gradient Boosting (XGBoost) model on tabular features extracted by SourceExtractor. While this XGBoost model approaches ASID-C in performance metrics, it does not offer the computational efficiency and reduced error propagation inherent in ASID-C’s direct image-based classification approach. ASID-C excels in low signal-to-noise ratio and crowded scenarios, potentially aiding in transient host identification and advancing deep-sky astronomy.

A contrastive learning framework for safety information extraction in construction

Typically named entity recognition (NER) and relation extraction (RE) from safety documentation (e.g., accident reports) adopt a pipeline processing approach whereby tasks are split into two sub-tasks. As a result, error propagation occurs between components, and useful information from one task may go unexploited by the other. Additionally, training sets to perform NER and RE from safety documentation are often limited and context-specific. Thus, our research addresses the following question: How can we accurately identify entities and extract relations from safety documentation using limited training sets? This paper utilizes ‘contrastive learning’ to tackle our research question. It proposes a contrastive learning-based cascade binary tagging framework (CasRel) to automatically and synchronously extract entities and relations from safety documents. A five-fold cross-validation process is used to validate the effectiveness and feasibility of our approach. Results from the validation process achieve an average precision of 77.8%, recall of 58.7%, and F1-score of 66.9%, outperforming CasRel with an increase of about 10% in precision, 5% in recall, and 7% in F1-score. Thus, our approach can accurately recognize entities and extract relations from safety documentation. The contributions of our study are twofold: (1) an improved unified model is developed to recognize and extract the entity and relation from safety documents to reduce error propagation and improve its accuracy; and (2) the concept of ‘contrastive learning’ is introduced in the design of the joint entity and relation extraction model with limited training sets.

JPEG Decoding With Nonlinear Inverse Transform Network and Progressive Recurrent Residual Network

JPEG is the most widely used image compression format, especially with the popularity of mobile or portable devices. However, the quality of a decoded JPEG image is usually degraded by compression artifacts such as blocking effect and ringing effect, especially at low bit rates. Recently, some convolutional neural network (CNN) based methods have been designed to solve the above problem. These methods take the problem as post-processing and only add a CNN-based post-processing network after a JPEG decoder to improve the image quality. In this paper, the JPEG decoding with nonlinear inverse transform network and progressive recurrent residual network (dubbed as JDNet) is proposed. JDNet can reconstruct JPEG images of different quality factors (QF) with only one model. In JDNet, first, the CNN-based inverse transform network (iTNet) is proposed to learn the nonlinear mapping from DCT coefficients to their corresponding original pixels, which is against the linear mapping in previous DCT networks. iTNet can reduce error propagation during inverse DCT and obtain more accurate reconstruction. Furthermore, iTNet can be combined with any JPEG post-processing method to improve its performance. Second, the progressive recurrent residual network (PRRN) is proposed for local feature extraction in the designed post-processing network which utilizes local and nonlocal similarities in multi-scale space (LNLMS) to further enhance the decoded image quality. The experimental results show that compared with JPEG, ARCNN, DnCNN and STRRN, the average gains of JDNet are 1.88dB, 0.63dB, 0.35dB, 0.24dB on Live1 dataset, 2.61dB, 1.11dB, 0.69dB and 0.29dB on Urban100 dataset, and 1.91dB, 0.69dB, 0.37dB and 0.13dB on BSD500 dataset, respectively.

Machine Learning With Data Assimilation and Uncertainty Quantification for Dynamical Systems: A Review

Data assimilation (DA) and uncertainty quantification (UQ) are extensively used in analysing and reducing error propagation in high-dimensional spatial-temporal dynamics. Typical applications span from computational fluid dynamics (CFD) to geoscience and climate systems. Recently, much effort has been given in combining DA, UQ and machine learning (ML) techniques. These research efforts seek to address some critical challenges in high-dimensional dynamical systems, including but not limited to dynamical system identification, reduced order surro-gate modelling, error covariance specification and model error correction. A large number of developed techniques and methodologies exhibit a broad applicability across numerous domains, resulting in the necessity for a comprehensive guide. This paper provides the first overview of state-of-the-art researches in this interdisciplinary field, covering a wide range of applications. This review is aimed at ML scientists who attempt to apply DA and UQ techniques to improve the accuracy and the interpretability of their models, but also at DA and UQ experts who intend to integrate cutting-edge ML approaches to their systems. Therefore, this article has a special focus on how ML methods can overcome the existing limits of DA and UQ, and vice versa. Some exciting perspectives of this rapidly developing research field are also discussed.

Turning Telecommunication Fiber-Optic Cables into Distributed Acoustic Sensors for Vibration-Based Bridge Health Monitoring

We introduce a nondedicated bridge health monitoring (BHM) system that turns pre-existing telecommunication fiber-optic cables into distributed acoustic sensors to collect bridge dynamic strain responses. Due to extensively installed telecommunication fiber cables in the cities, our telecommunication cable-based system enables efficient and low-cost BHM without the requirement of on-site sensor installation and maintenance; however, it is challenging to extract bridge damage-sensitive information (e.g., natural frequencies and mode shapes) from this nondedicated strain data as it has large measurement noise and error propagation. To overcome the challenge, we develop a physics-guided system identification method that models strain mode shapes based on physics-guided parametric mode shape functions derived from bridge dynamics. We then estimate the displacement mode shape function by analytically double-integrating the modeled strain mode shape. Our method improves the accuracy of estimating bridge damage-sensitive features and reduces error propagation by constraining strain and displacement mode shapes with bridge dynamics. We evaluated our system on a concrete continuous three-span bridge in San Jose, California. Our system successfully identified the first three natural frequencies and reconstructed strain and displacement mode shapes in a meter-scale resolution.

Document-level relation extraction with two-stage dynamic graph attention networks

Document-level Relation Extraction (RE) aims to infer complex semantic relations between entities in a document. Previous approaches leverage a multi-classification model to predict relation types between each entity pair. However, in contrast to sentence-level RE, document-level RE contains various entities expressed by mentions appearing across multiple sentences in a document. Therefore, the amount of negative instances (‘no relationship’) significantly outnumbers that of other positive instances in document-level RE. In addition, most existing methods construct static graphs with heuristic rules to capture the interactions among entities. However, these heuristic rules ignore the specificities of the documents. In this study, we propose a novel two-stage framework to extract document-level relations based on dynamic graph attention networks, namely TDGAT. In the first stage, we capture the relational links of the entity pairs using a binary classification model. In the second stage, we extract fine-grained relations among entities, including the type of ‘NA (no relationship)’. To reduce error propagation, we regard the entity pair links predicted in the first stage as the prior information and leverage them to reconstruct the document-level graphs of the second stage. In this manner, we can provide extra head and tail entity connection information for predicting relations in a document. Furthermore, we propose a dynamic graph strategy to explore the multi-hop interactions between related information. The experimental results show that our framework outperforms most existing models on the public document-level dataset DocRED. The extensive analysis demonstrates the effectiveness of our TDGAT in extracting inter-sentence relations.

Two-Stage Channel Estimation for RIS-Aided Multiuser mmWave Systems With Reduced Error Propagation and Pilot Overhead

Two-Stage Channel Estimation for RIS-Aided Multiuser mmWave Systems With Reduced Error Propagation and Pilot Overhead

Enhanced Motion Compensation for Deep Video Compression

Most of the existing deep learning-based video compression frameworks rely on motion estimation and compensation. However, the artifacts of the warped frames after motion compensation, which propagate the errors to the next frame, limit the video coding performance. In this work, we propose enhanced motion compensation for reduced error propagation in deep video compression. More specifically, we incorporate the designed convolutional neural network into Open DVC as the motion compensation enhancement network to remove noise in the predicted frame. With the enhanced frame, we jointly optimize the whole framework with a single loss function by considering the trade-off between bit cost and frame quality. Experiments show that the proposed enhanced motion compensation model reduces error propagation within a group of frames. Compared with Open DVC, our model can achieve 8.94% bit savings on average for standard test videos in terms of PSNR. Regarding MS-SSIM, our model outperforms Open DVC with 5.67% bit rate savings.

Towards Malay named entity recognition: an open-source dataset and a multi-task framework

Named entity recognition (NER) is a key component of many natural language processing (NLP) applications. The majority of advanced research, however, has not been widely applied to low-resource languages represented by Malay due to the data-hungry problem. In this paper, we present a system for building a Malay NER dataset (MS-NER) of 20,146 sentences through labelled datasets of homologous languages and iterative optimisation. Additionally, we propose a Multi-Task framework, namely MTBR, to integrate boundary information more effectively for NER. Specifically, boundary detection is treated as an auxiliary task and an enhanced Bidirectional Revision module with a gated ignoring mechanism is proposed to undertake conditional label transfer. This can reduce error propagation by the auxiliary task. We conduct extensive experiments on Malay, Indonesian, and English. Experimental results show that MTBR could achieve competitive performance and tends to outperform multiple baselines. The constructed dataset and model would be made available to the public as a new, reliable benchmark for Malay NER.

Power Distribution Based Beamspace Channel Estimation for mmWave Massive MIMO System With Lens Antenna Array

Millimeter-wave (mmWave) massive multiple-input multiple-output (MIMO) system with lens antenna array can achieve extremely high data rates with limited radio frequency (RF) chains. It brings challenges to channel estimation due to the gap between the number of RF chains and antennas. To solve this problem, by analyzing and exploiting the unique power distribution (PD) of the beamspace channel (BC) that differs from the general sparse signals, we propose a PD-based estimation scheme for the sparse BC. Specifically, we transform the issue into the direction of arrival (DOA) and complex gain estimation for each multipath component after PD-based support estimation. Meanwhile, we propose a method to reduce error propagation. Besides, we provide a lower bound for the error of the proposed PD-based scheme and explain the parameters that influence the performance. Finally, the numerical simulation results confirm the advantage of the proposed method over the conventional channel estimation ones in terms of accuracy and overhead. Meanwhile, the simulation results also prove the proposed proposition about the lower bound of normalized mean squared error.

Are Quantitative Errors Reduced with Time-of-Flight Reconstruction When Using Imperfect MR-Based Attenuation Maps for 18F-FDG PET/MR Neuroimaging?

We studied whether TOF reduces error propagation from attenuation correction to PET image reconstruction in PET/MR neuroimaging, by using imperfect attenuation maps in a clinical PET/MR system with 525 ps timing resolution. Ten subjects who had undergone 18F-FDG PET neuroimaging were included. Attenuation maps using a single value (0.100 cm−1) with and without air, and a 3-class attenuation map with soft tissue (0.096 cm−1), air and bone (0.151 cm−1) were used. CT-based attenuation correction was used as a reference. Volume-of-interest (VOI) analysis was conducted. Mean bias and standard deviation across the brain was studied. Regional correlations and concordance were evaluated. Statistical testing was conducted. Average bias and standard deviation were slightly reduced in the majority (23–26 out of 35) of the VOI with TOF. Bias was reduced near the cortex, nasal sinuses, and in the mid-brain with TOF. Bland–Altman and regression analysis showed small improvements with TOF. However, the overall effect of TOF to quantitative accuracy was small (3% at maximum) and significant only for two attenuation maps out of three at 525 ps timing resolution. In conclusion, TOF might reduce the quantitative errors due to attenuation correction in PET/MR neuroimaging, but this effect needs to be further investigated on systems with better timing resolution.