Semantic Dependency Research Articles

A bi-consolidating model for joint relational triple extraction

Current methods to extract relational triples directly make a prediction based on a possible entity pair in a raw sentence without depending on entity recognition. The task suffers from a serious semantic overlapping problem, in which several relation triples may share one or two entities in a sentence. In this paper, based on a two-dimensional sentence representation, a bi-consolidating model is proposed to address this problem by simultaneously reinforcing the local and global semantic features relevant to a relation triple. This model consists of a local consolidation component and a global consolidation component. The first component uses a pixel difference convolution to enhance semantic information of a possible triple representation from adjacent regions and mitigate noise in neighbouring neighbours. The second component strengthens the triple representation based a channel attention and a spatial attention, which has the advantage to learn remote semantic dependencies in a sentence. They are helpful to improve the performance of both entity identification and relation type classification in relation triple extraction. After evaluated on several publish datasets, the bi-consolidating model achieves competitive performance. Analytical experiments demonstrate the effectiveness of our model for relational triple extraction and give motivation for other natural language processing tasks.

DCS-YOLO: Defect detection model for new energy vehicle battery current collector.

The future trend in global automobile development is electrification, and the current collector is an essential component of the battery in new energy vehicles. Aiming at the misjudgment and omission caused by the confusing distribution, a wide range of sizes and types, and ambiguity of target defects in current collectors, an improved target detection model DCS-YOLO (DC-SoftCBAM YOLO) based on YOLOv5 is proposed. Firstly, the detection rate of defects with different scales is improved by adding detection layers; Secondly, we use the designed DC module as the backbone network to help the model capture the global information and semantic dependencies of the target, and effectively improve the generalization ability and detection performance of the model. Finally, in the neck part, we integrate our designed Convolutional Block Attention Module (SoftPool Convolutional Block Attention Module, SoftCBAM), which can adaptively learn the importance of channels, enhance feature representation, and enable the model to better deal with target details. Experimental results show that the mAP50 of the proposed DCS-YOLO model is 92.2%, which is 5.1% higher than the baseline model. The FPS reaches 147.1, and the detection accuracy of various defect categories is improved, especially Severely bad and No cover, and the detection recall rate reaches 100%. This method has high target detection model efficiency and meets the requirements of real-time detection of battery collector defects.

Adaptive Taxonomy Learning and Historical Patterns Modeling for Patent Classification

Patent classification aims to assign multiple International Patent Classification (IPC) codes to a given patent. Existing methods for automated patent classification primarily focus on analyzing the text descriptions of patents. However, apart from the textual information, each patent is also associated with some assignees, and the knowledge of their previously applied patents can often be valuable for accurate classification. Furthermore, the hierarchical taxonomy defined by the IPC system provides crucial contextual information and enables models to leverage the correlations between IPC codes for improved classification accuracy. However, existing methods fail to incorporate the above aspects and lead to reduced performance. To address these limitations, we propose an integrated framework that comprehensively considers patent-related information for patent classification. To be specific, we first present an IPC codes correlations learning module to capture both horizontal and vertical information within the IPC codes. This module effectively captures the correlations by adaptively exchanging and aggregating messages among IPC codes at the same level (horizontal information) and from both parent and children codes (vertical information), which allows for a comprehensive integration of knowledge and relationships within the IPC hierarchical taxonomy. Additionally, we design a historical application patterns learning component to incorporate previous patents of the corresponding assignee by aggregating high-order temporal information via a dual-channel graph neural network. Finally, our approach combines the contextual information from patent texts, which encompasses the semantics of IPC codes, with assignees’ sequential preferences to make predictions. Experimental evaluations on real-world datasets demonstrate the superiority of our proposed approach over existing methods. Moreover, we present the model’s ability to capture the temporal patterns of assignees and the semantic dependencies among IPC codes.

Piecewise convolutional neural network relation extraction with self-attention mechanism

The task of relation extraction in natural language processing is to identify the relation between two specified entities in a sentence. However, the existing model methods do not fully utilize the word feature information and pay little attention to the influence degree of the relative relation extraction results of each word. In order to address the aforementioned issues, we propose a relation extraction method based on self-attention mechanism (SPCNN-VAE) to solve the above problems. First, we use a multi-head self-attention mechanism to process word vectors and generate sentence feature vector representations, which can be used to extract semantic dependencies between words in sentences. Then, we introduce the word position to combine the sentence feature representation with the position feature representation of words to form the input representation of piecewise convolutional neural network (PCNN). Furthermore, to identify the word feature information that is most useful for relation extraction, an attention-based pooling operation is employed to capture key convolutional features and classify the feature vectors. Finally, regularization is performed by a variational autoencoder (VAE) to enhance the encoding ability of model word information features. The performance analysis is performed on SemEval 2010 task 8, and the experimental results show that the proposed relation extraction model is effective and outperforms some competitive baselines.

SIAT: Document-level Event Extraction via Spatiality-Augmented Interaction Model with Adaptive Thresholding

Document-level event extraction endeavors to automatically extract structural events from a given document. Many existing approaches focus on modeling entity interactions and decoding these interactions into events, assigning each entity as an event argument. However, these approaches encounter two primary limitations: they exclusively capture semantic dependencies to model entity interactions, overlooking the indication of the spatial distribution features of entities; they decode interactions imprecisely with a hard binary-classification boundary, potentially failing to calibrate micro differences in interactions. To overcome these limitations, we introduce a novel approach termed the S patiality-augmented I nteraction Model with A daptive T hresholding (SIAT). Our method addresses the first limitation by calculating the relative position encoding of entities to represent spatial interaction features. These features are then integrated with multi-granularity semantic interactions, enhancing the modeling of entity interactions for each entity pair. Furthermore, we introduce an adaptive event decoding mechanism, which establishes a more flexible decision boundary for different entity interactions. Additionally, an adaptive loss function for threshold learning is designed to further refine the model. Experimental results demonstrate that our proposed method achieves competitive performance compared to state-of-the-art methods on two public event extraction datasets while maintaining considerable training efficiency.

BAMRE: Joint extraction model of Chinese medical entities and relations based on Biaffine transformation with relation attention

Electronic Health Records (EHRs) contain various valuable medical entities and their relationships. Although the extraction of biomedical relationships has achieved good results in the mining of electronic health records and the construction of biomedical knowledge bases, there are still some problems. There may be implied complex associations between entities and relationships in overlapping triplets, and ignoring these interactions may lead to a decrease in the accuracy of entity extraction. To address this issue, a joint extraction model for medical entity relations based on a relation attention mechanism is proposed. The relation extraction module identifies candidate relationships within a sentence. The attention mechanism based on these relationships assigns weights to contextual words in the sentence that are associated with different relationships. Additionally, it extracts the subject and object entities. Under a specific relationship, entity vector representations are utilized to construct a global entity matching matrix based on Biaffine transformations. This matrix is designed to enhance the semantic dependencies and relational representations between entities, enabling triplet extraction. This allows the two subtasks of named entity recognition and relation extraction to be interrelated, fully utilizing contextual information within the sentence, and effectively addresses the issue of overlapping triplets.Experimental observations from the CMeIE Chinese medical relation extraction dataset and the Baidu2019 Chinese dataset confirm that our approach yields the superior F1 score across all cutting-edge baselines. Moreover, it offers substantial performance improvements in intricate situations involving diverse overlapping patterns, multitudes of triplets, and cross-sentence triplets.

An entity alignment method with attribute augmentation and contrastive learning

Abstract The entity alignment(EA) task is to identify entities with the same semantics in the knowledge graph(KG), an essential issue in KG fusion and big data mining. Existing entity alignment methods mainly adopt graph embedding-based methods. However, they still have some shortcomings. First, they heavily rely on high-quality alignment seed and external semantic information. Secondly, the present attention mechanism focuses on the entire graph information, neglecting the noise of attribute information. This paper proposes an EA approach based on Attribute Augmentation and Contrastive Learning (AACL). Our method introduces attribute augmentation to enhance the structure information of knowledge graphs and reduce dependence on alignment seed. A masked attention mechanism is developed to emphasize important attribute information and mask out invalid attributes to better capture semantic dependencies in the KG. Experimental results on three public datasets indicate that our AACL outperforms the present entity alignment approaches.

Motion-guided semantic alignment for line art animation colorization

Line art animation colorization aims to colorize a sequence of line art frames with a reference while ensuring semantic alignment across frames. Recent methods have utilized similarity metrics to maintain inter-frame semantic alignment. However, these approaches typically struggle with precise pixel-to-pixel motion alignment, resulting in issues like semantic inconsistencies and color blurring. To address these problems, we introduce an innovative Motion-Guided Colorization Network(MGC-Net), which incorporates line art motions into account for achieving coarse to fine semantic alignment. At the coarse level, our approach includes a Hierarchical Cost-feature Distillation (HCD) module. This module is designed to establish accurate pixel-to-pixel motion alignment between frames. It employs a motion distillation loss, which effectively transfers accurate color motion representation to line arts at varying hierarchical levels. Specifically, we utilize the cost features in inter-frames as a representation of motion, processed through our self-supervision operator. At the finer level, we propose the Correlation Refinement (CR) module. This module enhances semantic alignment by integrating semantic contextual dependencies between the color reference and the motion-based reference obtained from our motion-based confidence estimation block. CR module refines the final colorization process, improving clear color detail and semantic consistency across frames. Quantitative and qualitative experiments on animation datasets show that MGC-Net outperforms the state-of-the-art methods.

Fusing temporal and semantic dependencies for session-based recommendation

Session-based recommendation (SBR) predicts the next item in user sequences. Existing research focuses on item transition patterns, neglecting semantic information dependencies crucial for understanding users’ preferences. Incorporating semantic characteristics is vital for accurate recommendations, especially in applications like user purchase sequences. In this paper, to tackle the above issue, we novelly propose a framework that hierarchically fuses temporal and semantic dependencies. Technically, we present the Item Transition Dependency Module and Semantic Dependency Module based on the whole session set: (i) Item Transition Dependency Module is exclusively to learn the item embeddings through temporal relations and utilizes item transitions from both global and local levels; (ii) Semantic Dependency Module develops mutually independent embeddings of both sessions and items via stable interaction relations. In addition, under the unified organization of the Cross View, semantic information is adaptively incorporated into the temporal dependency learning and used to improve the performance of SBR. Extensive experiments on three large-scale real-world datasets show the superiority of our framework over current state-of-the-art methods. In particular, our model improves its performance over SOTA on all three datasets, with 5.5%, 0.2%, and 3.0% improvements on Recall@20, and 5.8%, 4.6%, and 2.0% improvements on MRR@20, respectively.

A global contextual enhanced structural-aware transformer for sequential recommendation

Sequential recommendation (SR) has become a research hotspot recently. In our research, we observe that most existing SR models only leverage each user’s own interaction sequence to make recommendation. We argue that leveraging global contextual information across different interaction sequences could enrich item representations and thereby improve recommendation performance. To achieve this, we formulate a global graph from different sequences, providing global contextual information for each sequence. Specifically, we propose to conduct graph contrastive learning on a subgraph sampled from the global graph and a local sequence graph built from each sequence to augment item representations within each sequence. At the same time, we observe that structural dependencies, referring to relationships between items based on the graphic structure, can be extracted from the constructed global graph. Capturing structural dependencies between items may enrich the item representations. To leverage structural dependencies, we propose a new attention mechanism referred to as the Jaccard attention. While prevalent Transformer-based SR models capture semantic dependencies, referring to relationships between items based on item embeddings, in a sequence through self-attention. Therefore, it is beneficial to capture both semantic and structural dependencies between items in a sequence to further enrich item representations. Specifically, we employ two sequence encoders based on the self-attention and the proposed Jaccard attention to capture semantic and structural dependencies between items in a sequence, respectively. Overall, we propose a Global Contextual enhanced Structural-aware Transformer (GC-ST) for SR. Extensive experiments carried out on three widely used datasets demonstrate the effectiveness of GC-ST.

Category-sensitive semi-supervised semantic segmentation framework for land-use/land-cover mapping with optical remote sensing images

High-quality land-use/land-cover mapping with optical remote sensing images yet presents significant work. Even though fully convolutional semantic segmentation models have recently contributed to popular solutions, the lack of annotation data may lead to severe degradations in their inference performance. Besides, the category confusion in high-resolution representations will further exacerbate the adverse effects. In this paper, we propose a category-sensitive semi-supervised semantic segmentation framework to address these weaknesses by employing massive unlabeled data. With the perturbations from adopted hybrid data augmentation structures, we first focus on the output space and execute regularization constraints to learn category-specific discriminative features. It is formulated with a consistency self-training procedure where a dynamic class-balanced threshold selection scheme is proposed to provide high-confident pseudo supervisions for each category. In addition, we introduce pixel-wise contrastive learning on the common embedding space from both labeled and unlabeled data domains to further facilitate the semantic dependencies among category features, in which the reliable labels are leveraged as guidance for pixel sample selection. We verify the proposed framework on two benchmark land-use/land-cover datasets, and the experimental results demonstrate its competitive performance to other state-of-the-art semi-supervised methods.

CC-De-YOLO: A Multiscale Object Detection Method for Wafer Surface Defect

Abstract Surface defect detection on wafers is crucial for quality control in semiconductor manufacturing. However, the complexity of defect spatial features, including mixed defect types, large scale differences, and overlapping, results in low detection accuracy. In this paper, we propose a CC-De-YOLO model, which is based on the YOLOv7 backbone network. Firstly, the coordinate attention is inserted into the main feature extraction network. Coordinate attention decomposes channel attention into two one-dimensional feature coding processes, which are aggregated along both horizontal and vertical spatial directions to enhance the network’s sensitivity to orientation and position. Then, the nearest neighbor interpolation in the upsampling part is replaced by the CAR-EVC module, which predicts the upsampling kernel from the previous feature map and integrates semantic information into the feature map. Two residual structures are used to capture long-range semantic dependencies and improve feature representation capability. Finally, an efficient decoupled detection head is used to separate classification and regression tasks for better defect classification. To evaluate our model’s performance, we established a wafer surface defect dataset containing six typical defect categories. The experimental results show that the CCDe-YOLO model achieves 91.0% mAP@0.5 and 46.2% mAP@0.5:0.95, with precision of 89.5% and recall of 83.2%. Compared with the original YOLOv7 model and other object detection models, CC-De-YOLO performs better. Therefore, our proposed method meets the accuracy requirements for wafer surface defect detection and has broad application prospects. The dataset containing surface defect data on wafers is currently publicly available on GitHub (https://github.com/ztao3243/Wafer-Datas.git).

Semantic dependency and local convolution for enhancing naturalness and tone in text-to-speech synthesis

Self-attention-based networks have become increasingly popular due to their exceptional performance in parallel training and global context modeling. However, it may fall short of capturing local dependencies, particularly in datasets with strong local correlations. To address this challenge, we propose a novel method that utilizes semantic dependency to extract linguistic information from the original text. The semantic relationship between nodes serves as prior knowledge to refine the self-attention distribution. Additionally, to better fuse local contextual information, we introduce a one-dimensional convolution neural network to generate the query and value matrices in the self-attention mechanism, taking advantage of the strong correlation between input characters. We apply this variant of the self-attention network to text-to-speech tasks and propose a non-autoregressive neural text-to-speech model. To enhance pronunciation accuracy, we separate tones from phonemes as independent features in model training. Experimental results show that our model yields good performance in speech synthesis. Specifically, the proposed method significantly improves the processing of pause, stress, and intonation in speech.

Adaptive feature extraction for entity relation extraction

Effective capturing of semantic dependencies within sentences is pivotal to support relation extraction. However, challenges such as feature sparsity, and the complexity of identifying the structure of target entity pairs brought by the traditional methods of feature extraction pose significant obstacles for relation extraction. Existing methods that rely on combined features or recurrent networks also face limitations, such as over-reliance on prior knowledge or the gradient vanishing problem. To address these limitations, we propose an Adaptive Feature Extraction (AFE) method, combining neural networks with feature engineering to capture high-order abstract and long-distance semantic dependencies. Our approach extracts atomic features from sentences, maps them into distributed representations, and categorizes these representations into multiple mixed features through adaptive combination, setting it apart from other methods. The proposed AFE-based model uses four different convolutional layers to facilitate feature learning and weighting from the adaptive feature representations, thereby enhancing the discriminative power of deep networks for relation extraction. Experimental results on the English datasets ACE05 English, SciERC, and the Chinese datasets ACE05 Chinese, and CLTC(SanWen) demonstrated the superiority of our method, the F1 scores were improved by 4.16%, 3.99%, 0.82%, and 1.60%, respectively. In summary, our AFE method provides a flexible, and effective solution to some challenges in cross-domain and cross-language relation extraction.

HindiPersonalityNet: Personality Detection in Hindi Conversational Data Using Deep Learning with Static Embedding

Personality detection along with other behavioral and cognitive assessment can essentially explain why people act the way they do and can be useful to various online applications such as recommender systems, job screening, matchmaking, and counseling. Additionally, psychometric natural language processing relying on textual cues and distinctive markers in writing style within conversational utterances reveals signs of individual personalities. This work demonstrates a text-based deep neural model, HindiPersonalityNet, of classifying conversations into three personality categories (ambivert, extrovert, introvert) for detecting personality in Hindi conversational data. The model utilizes a gated recurrent unit with BioWordVec embeddings for text classification and is trained/tested on a novel dataset, शख्सियत (pronounced as Shakhsiyat) curated using dialogues from an Indian crime-thriller drama series, Aarya . The model achieves an F1-score of 0.701 and shows the potential for leveraging conversational data from various sources to understand and predict a person's personality traits. It exhibits the ability to capture both semantic and long-distance dependencies in conversations and establishes the effectiveness of our dataset as a benchmark for personality detection in Hindi dialogue data. Further, a comprehensive comparison of various static and dynamic word embedding is done on our standardized dataset to ascertain the most suitable embedding method for personality detection.

Fine-grained sentiment analysis using multidimensional feature fusion and GCN

ABSTRACT The demand for analysing sentiment information in social media data is increasing. However, current fine-grained sentiment analysis methods fail to consider both global and local semantic features simultaneously, leading to the oversight of grammatical information in sentences and an inability to address the issue of polysemy. To address these challenges, we propose a microblog fine-grained sentiment analysis model based on multidimensional feature fusion and graph convolutional neural networks (GCN). Built upon the ALBERT model, we utilize BiLSTM and capsule network models to extract global and local semantic features, thereby capturing bidirectional semantic dependencies and textual positional semantic information. Finally, we employ multi-head self-attention and GCN to select key features and sentence information, ensuring the integrity of fine-grained features. The experimental results indicate that the model outperforms several other models on fine-grained sentiment analysis datasets SMP2020-EWECT, NLPCC2013, NLPCC2014, and the binary classification dataset weibo_senti_100k, achieving accuracies of 80.64%, 67.19%, 71.37%, and 98.43%, respectively.

A Fine-Grained Sentiment Analysis Method Using Transformer for Weibo Comment Text

Many existing fine-grained sentiment analysis (FGSA) methods have problems such as easy loss of fine-grained information, difficulty in solving polysemy and imbalanced sample categories. Therefore, a Transformer based FGSA method for Weibo comment text is proposed. Firstly, the RoBERTa model with knowledge augmentation was used to dynamically encode the text so as to solving the polysemy issue. Then, BiLSTM is used to effectively capture bidirectional global semantic dependency features. Next, Transformer is used to fuse multi-dimensional features and adaptively strengthen key features to overcome the problem of fine-grained information loss. Finally, an improved Focal Loss function is utilized for training to solve the issue of imbalanced sample categories. As demonstrated by the experimental outcomes on the SMP2020-EWECT, NLPCC 2013 Task 2, NLPCC 2014 Task 1, and weibo_senti_100k datasets, the suggested method outperforms the alternatives for advanced comparison methods.

Event-driven weakly supervised video anomaly detection

Inspired by the observations of human working manners, this work proposes an event-driven method for weakly supervised video anomaly detection. Complementary to the conventional snippet-level anomaly detection, this work designs an event analysis module to predict the event-level anomaly scores as well. It first generates event proposals simply via a temporal sliding window and then constructs a cascaded causal transformer to capture temporal dependencies for potential events of varying durations. Moreover, a dual-memory augmented self-attention scheme is also designed to capture global semantic dependencies for event feature enhancement. The network is learned with a standard multiple instance learning (MIL) loss, together with normal-abnormal contrastive learning losses. During inference, the snippet- and event-level anomaly scores are fused for anomaly detection. Experiments show that the event-level analysis helps to detect anomalous events more continuously and precisely. The performance of the proposed method on three public datasets demonstrates that the proposed approach is competitive with state-of-the-art methods.

TS-CAM: Token Semantic Coupled Attention Map for Weakly Supervised Object Localization.

Weakly supervised object localization (WSOL), which trains object localization models using solely image category annotations, remains a challenging problem. Existing approaches based on convolutional neural networks (CNNs) tend to miss full object extent while activating discriminative object parts. Based on our analysis, this is caused by CNN's intrinsic characteristics, which experiences difficulty to capture object semantics at long distances. In this article, we introduce the vision transformer to WSOL, with the aim to capture long-range semantic dependency of features by leveraging transformer's cascaded self-attention mechanism. We propose the token semantic coupled attention map (TS-CAM) method, which first decomposes class-aware semantics and then couples the semantics with attention maps for semantic-aware activation. To capture object semantics at long distances and avoid partial activation, TS-CAM performs spatial embedding by partitioning an image to a set of patch tokens. To incorporate object category information to patch tokens, TS-CAM reallocates category-related semantics to each patch token. The patch tokens are finally coupled with attention maps which are semantic-agnostic to perform semantic-aware object localization. By introducing semantic tokens to produce semantic-aware attention maps, we further explore the capability of TS-CAM for multicategory object localization. Experiments show that TS-CAM outperforms its CNN-CAM counterpart by 11.6% and 28.9% on ILSVRC and CUB-200-2011 datasets, respectively, improving the state-of-the-art with large margins. TS-CAM also demonstrates superiority for multicategory object localization on the Pascal VOC dataset. The code is available at github.com/yuanyao366/ts-cam-extension.

Learning multi-graph structure for Temporal Knowledge Graph reasoning

Temporal Knowledge Graph (TKG) reasoning aiming at forecasting future events based on historical snapshots distributed over timestamps. Referred to as extrapolation in numerous studies, this aspect of research in KG has garnered significant attention in recent years. Owing to its extreme versatility and variation in spatial and temporal correlations, TKG reasoning presents a challenging task, demanding efficient capture of concurrent structures and evolutional interactions among facts. While existing methods have made strides in this direction, they still fall short of harnessing the diverse forms of intrinsic expressive semantics of TKGs, which encompass entity correlations across multiple timestamps and periodicity of temporal information. This limitation constrains their ability to thoroughly reflect historical dependencies and future trends. In response to these drawbacks, this paper proposes an innovative reasoning approach that focuses on Learning Multi-graph Structure (LMS). Concretely, it comprises three distinct modules concentrating on multiple aspects of graph structure knowledge within TKGs, including concurrent and evolutional patterns along timestamps, query-specific correlations across timestamps, and semantic dependencies of timestamps, which capture TKG features from various perspectives. Besides, LMS incorporates an adaptive gate for merging entity representations both along and across timestamps effectively. Moreover, it integrates timestamp semantics into graph attention calculations and time-aware decoders, in order to impose temporal constraints on events and narrow down prediction scopes with historical statistics. Extensive experimental results on five event-based benchmark datasets demonstrate that LMS outperforms state-of-the-art extrapolation models, indicating the superiority of modeling a multi-graph perspective for TKG reasoning.