Commonsense Knowledge Graph Research Articles

Triple confidence-aware encoder–decoder model for commonsense knowledge graph completion

Triple confidence-aware encoder–decoder model for commonsense knowledge graph completion

Acquiring and modeling abstract commonsense knowledge via conceptualization

Conceptualization, or viewing entities and situations as instances of abstract concepts in mind and making inferences based on that, is a vital component in human intelligence for commonsense reasoning. Despite recent progress in artificial intelligence to acquire and model commonsense attributed to neural language models and commonsense knowledge graphs (CKGs), conceptualization is yet to be introduced thoroughly, making current approaches ineffective to cover knowledge about countless diverse entities and situations in the real world. To address the problem, we thoroughly study the role of conceptualization in commonsense reasoning, and formulate a framework to replicate human conceptual induction by acquiring abstract knowledge about events regarding abstract concepts, as well as higher-level triples or inferences upon them. We then apply the framework to ATOMIC, a large-scale human-annotated CKG, aided by the taxonomy Probase. We annotate a dataset on the validity of contextualized conceptualizations from ATOMIC on both event and triple levels, develop a series of heuristic rules based on linguistic features, and train a set of neural models to generate and verify abstract knowledge. Based on these components, a pipeline to acquire abstract knowledge is built. A large abstract CKG upon ATOMIC is then induced, ready to be instantiated to infer about unseen entities or situations. Finally, we empirically show the benefits of augmenting CKGs with abstract knowledge in downstream tasks like commonsense inference and zero-shot commonsense QA.

Amplifying Diversity and Quality in Commonsense Knowledge Graph Completion (Student Abstract)

Conventional commonsense knowledge graph completion (CKGC) methods provide inadequate sequence when fine-tuning or generating stages and incorporate full fine-tuning, which fail to align with the autoregressive model's pre-training patterns and have insufficient parameter efficiency. Moreover, decoding through beam or greedy search produces low diversity and high similarity in generated tail entities. Hence, we resort to prefix-tuning and propose a lightweight, effective pipeline to enhance the quality and diversity of extracted commonsense knowledge. Precisely, we measure head entity similarity to yield and then concatenate top-k tuples before each target tuple for prefix-tuning the source LM, thereby improving the efficiency and speed for pretrained models; then, we design a penalty-tailored diverse beam search (p-DBS) for decoding tail entities, producing a greater quantity and diversity of generated commonsense tuples; besides, a filter strategy is utilized to filter out invalid commonsense knowledge. Through extensive automatic evaluations, including ChatGPT scoring, our method can extract diverse, novel, and accurate commonsense knowledge (CK).

TodBR: Target-Oriented Dialog with Bidirectional Reasoning on Knowledge Graph

Target-oriented dialog explores how a dialog agent connects two topics cooperatively and coherently, which aims to generate a “bridging” utterance connecting the new topic to the previous conversation turn. The central focus of this task entails multi-hop reasoning on a knowledge graph (KG) to achieve the desired target. However, current target-oriented dialog approaches suffer from inefficiencies in reasoning and the inability to locate pertinent key information without bidirectional reason. To address these limitations, we present a bidirectional reasoning model for target-oriented dialog implemented on a commonsense knowledge graph. Furthermore, we introduce an automated technique for constructing dialog subgraphs, which aids in acquiring multi-hop reasoning capabilities. Our experiments demonstrate that our proposed method attains superior performance in reaching the target while providing more coherent responses.

Long Dialogue Emotion Detection Based on Commonsense Knowledge Graph Guidance

Dialogue emotion detection is always challenging due to human subjectivity and the randomness of dialogue content. In a conversation, the emotion of each person often develops via a cumulative process, which can be influenced by many elements of uncertainty. Much commonsense knowledge influences people's emotions imperceptibly, such as experiential or habitual knowledge. In the process of conversation, this commonsense knowledge information can be used to enrich the semantic information of each utterance and improve the accuracy of emotion recognition. In this paper, we propose a growing graph model for dialogues emotion detection based on retrieval of external knowledge atlas ATOMIC from local and global respectively, which can effectively represent the dialogues as a process variable in a sequence and the correlation among utterances also can be represented by the graph model. In particular, 1) we introduce a common sense knowledge graph for linking the commonsense knowledge retrieved from external knowledge atlas ATOMIC, which can effectively add auxiliary information to improve the performance of each utterance's representation. 2) We propose a novel self-supervised learning method for extracting the latent topic of each dialogue. Based on this design, we also propose an effective optimization mechanism to make the representation (embedding) of latent topic has a better distinction for the next operation. 3) Finally, the cross-attention module is utilized to combine the utterances' features and the latent conversation topic information. The attention mechanism can effectively use topic information to supplement the representation of utterances and improve recognition performance. The model is tested on three popular datasets in dialogue emotion detection and is empirically demonstrated to outperform the state-of-the-art approaches. Meanwhile, to demonstrate the performance of our approach, we also build a long dialogue dataset. The average length of each conversation is over 50 utterances. The final experimental results also demonstrate the superior performance of our approach.

Identify Event Causality with Knowledge and Analogy

Event causality identification (ECI) aims to identify the causal relationship between events, which plays a crucial role in deep text understanding. Due to the diversity of real-world causality events and difficulty in obtaining sufficient training data, existing ECI approaches have poor generalizability and struggle to identify the relation between seldom seen events. In this paper, we propose to utilize both external knowledge and internal analogy to improve ECI. On the one hand, we utilize a commonsense knowledge graph called ConceptNet to enrich the description of an event sample and reveal the commonalities or associations between different events. On the other hand, we retrieve similar events as analogy exam- ples and glean useful experiences from such analogous neigh- bors to better identify the relationship between a new event pair. By better understanding different events through exter- nal knowledge and making an analogy with similar events, we can alleviate the data sparsity issue and improve model gener- alizability. Extensive evaluations on two benchmark datasets show that our model outperforms other baseline methods by around 18% on the F1-value on average

An Interpretable Fake News Detection Method Based on Commonsense Knowledge Graph

Existing deep learning-based methods for detecting fake news are uninterpretable, and they do not use external knowledge related to the news. As a result, the authors of the paper propose a graph matching-based approach combined with external knowledge to detect fake news. The approach focuses on extracting commonsense knowledge from news texts through knowledge extraction, extracting background knowledge related to news content from a commonsense knowledge graph through entity extraction and entity disambiguation, using external knowledge as evidence for news identification, and interpreting the final identification results through such evidence. To achieve the identification of fake news containing commonsense errors, the algorithm uses random walks graph matching and compares the commonsense knowledge embedded in the news content with the relevant external knowledge in the commonsense knowledge graph. The news is then discriminated as true or false based on the results of the comparative analysis. From the experimental results, the method can achieve 91.07%, 85.00%, and 89.47% accuracy, precision, and recall rates, respectively, in the task of identifying fake news containing commonsense errors.

MKBQA: Question Answering over Knowledge Graph Based on Semantic Analysis and Priority Marking Method

In the field of question answering-based knowledge graphs, due to the complexity of the construction of knowledge graphs, a domain-specific knowledge graph often cannot contain some common-sense knowledge, which makes it impossible to answer questions that involve common-sense and domain knowledge at the same time. Therefore, this study proposes a knowledge graph-based question answering method in the computer science domain, which facilitates obtaining complete answers in this domain. In order to solve the problem of natural language problems being difficult to match with structured knowledge, a series of logic rules are first designed to convert natural language into triples of the question. Then, a semantic query expansion strategy based on WordNet is proposed and a priority marking algorithm is proposed to mark the order of triples of the question. Finally, when a question triple corresponds to multiple triples in the knowledge graph, it can be solved by the proposed SimCSE-based similarity method. The designed logic rules can deal with each type of question in a targeted manner according to the different question words and can effectively transform the question text into question triples. In addition, the proposed priority marking algorithm can effectively mark the order in the triple of the question. MKBQA can answer not only computer science-related questions but also extended open domain questions. In practical applications, answering a domain question often cannot rely solely on one knowledge graph. It is necessary to combine domain knowledge and common-sense knowledge. The MKBQA method provides a new idea and can be easily migrated from the field of computer science to other fields. Experiment results on real-world data sets show that, as compared to baselines, our method achieves significant improvements to question answering and can combine common-sense and domain-specific knowledge graphs to give a more complete answer.

Understanding Chinese Moral Stories with Further Pre-Training

The goal of moral understanding is to grasp the theoretical concepts embedded in a narrative by delving beyond the concrete occurrences and dynamic personas. Specifically, the narrative is compacted into a single statement without involving any characters within the original text, necessitating a more astute language model that can comprehend connotative morality and exhibit commonsense reasoning. The “pretraining + fine-tuning” paradigm is widely embraced in neural language models. In this paper, we propose an intermediary phase to establish an improved paradigm of “pre-training + further pre-training + finetuning”. Further pre-training generally refers to continual learning on task-specific or domain-relevant corpora before being applied to target tasks, which aims at bridging the gap in data distribution between the phases of pre-training and fine-tuning. Our work is based on a Chinese dataset named STORAL-ZH that composes of 4k human-written story-moral pairs. Furthermore, we design a two-step process of domain-adaptive pre-training in the intermediary phase. The first step depends on a newly-collected Chinese dataset of Confucian moral culture. And the second step bases on the Chinese version of a frequently-used commonsense knowledge graph (i.e. ATOMIC) to enrich the backbone model with inferential knowledge besides morality. By comparison with several advanced models including BERTbase, RoBERTa-base and T5-base, experimental results on two understanding tasks demonstrate the effectiveness of our proposed three-phase paradigm.

Enhancing aspect-category sentiment analysis via syntactic data augmentation and knowledge enhancement

The goal of aspect-category sentiment analysis (ACSA) is to predict the sentiment polarity toward a specific aspect category from reviewers’ expressed opinions in a sentence. With the boom of pretrained language models, various relative methods have achieved significant improvements in ACSA. However, two major issues still remain to be solved. First, most of these studies usually follow the canonical method of fine-tuning on limited labeled data, neglecting to leverage external knowledge to further enhance ACSA performance. Second, aspect categories are usually abstract concepts that are mentioned explicitly or implicitly, and the corresponding different sentiment polarities are not easy to accurately recognize. To address these issues, we first transform the ACSA task into a sentence-pair classification task with natural language inference, constructing synthetic sentences as hypotheses based on the predefined aspect categories and the prompt-generation sentence template. Then the model applies a passivization transformation to the synthetic sentences and generates more syntactic data to augment the limited training data. Furthermore, we enhance ACSA with curated knowledge from a common sense knowledge graph. Finally, different representations are synergistically fused with a gating mechanism to output richer sentiment features and enable context-, syntax-, and knowledge-aware predictions. Experimental results on three challenging benchmark datasets show that the proposed model outperforms some competitive baselines.

Improving FMEA Comprehensibility via Common-Sense Knowledge Graph Completion Techniques

Improving FMEA Comprehensibility via Common-Sense Knowledge Graph Completion Techniques

Aspect Sentiment Triplet Extraction Incorporating Syntactic Constituency Parsing Tree and Commonsense Knowledge Graph

Aspect Sentiment Triplet Extraction Incorporating Syntactic Constituency Parsing Tree and Commonsense Knowledge Graph

Dynamic memory supported dialog generation model based on commonsense knowledge graph

Dynamic memory supported dialog generation model based on commonsense knowledge graph

Utilizing Language Models to Expand Vision-Based Commonsense Knowledge Graphs

The introduction and ever-growing size of the transformer deep-learning architecture have had a tremendous impact not only in the field of natural language processing but also in other fields. The transformer-based language models have contributed to a renewed interest in commonsense knowledge due to the abilities of deep learning models. Recent literature has focused on analyzing commonsense embedded within the pre-trained parameters of these models and embedding missing commonsense using knowledge graphs and fine-tuning. We base our current work on the empirically proven language understanding of very large transformer-based language models to expand a limited commonsense knowledge graph, initially generated only on visual data. The few-shot-prompted pre-trained language models can learn the context of an initial knowledge graph with less bias than language models fine-tuned on a large initial corpus. It is also shown that these models can offer new concepts that are added to the vision-based knowledge graph. This two-step approach of vision mining and language model prompts results in the auto-generation of a commonsense knowledge graph well equipped with physical commonsense, which is human commonsense gained by interacting with the physical world. To prompt the language models, we adapted the chain-of-thought method of prompting. To the best of our knowledge, it is a novel contribution to the domain of the generation of commonsense knowledge, which can result in a five-fold cost reduction compared to the state-of-the-art. Another contribution is assigning fuzzy linguistic terms to the generated triples. The process is end to end in the context of knowledge graphs. It means the triples are verbalized to natural language, and after being processed, the results are converted back to triples and added to the commonsense knowledge graph.

Improving Zero-Shot Learning Baselines with Commonsense Knowledge

Zero-shot learning — the problem of training and testing on a completely disjoint set of classes — relies greatly on its ability to transfer knowledge from train classes to test classes. Traditionally semantic embeddings consisting of human-defined attributes or distributed word embeddings are used to facilitate this transfer by improving the association between visual and semantic embeddings. In this paper, we take advantage of explicit relations between nodes defined in ConceptNet, a commonsense knowledge graph, to generate commonsense embeddings of the class labels by using a graph convolution network-based autoencoder. Our experiments performed on three standard benchmark datasets surpass the strong baselines when we fuse our commonsense embeddings with existing semantic embeddings, i.e., human-defined attributes and distributed word embeddings. This work paves the path to more brain-inspired approaches to zero-short learning.

Enhanced Story Comprehension for Large Language Models through Dynamic Document-Based Knowledge Graphs

Large transformer-based language models have achieved incredible success at various tasks which require narrative comprehension, including story completion, answering questions about stories, and generating stories ex nihilo. However, due to the limitations of finite context windows, these language models struggle to produce or understand stories longer than several thousand tokens. In order to mitigate the document length limitations that come with finite context windows, we introduce a novel architecture that augments story processing with an external dynamic knowledge graph. In contrast to static commonsense knowledge graphs which hold information about the real world, these dynamic knowledge graphs reflect facts extracted from the story being processed. Our architecture uses these knowledge graphs to create information-rich prompts which better facilitate story comprehension than prompts composed only of story text. We apply our architecture to the tasks of question answering and story completion. To complement this line of research, we introduce two long-form question answering tasks, LF-SQuAD and LF-QUOREF, in which the document length exceeds the size of the language model's context window, and introduce a story completion evaluation method that bypasses the stochastic nature of language model generation. We demonstrate broad improvement over typical prompt formulation methods for both question answering and story completion using GPT-2, GPT-3 and XLNet.

CSKG: The Commonsense Knowledge Graph

CSKG: The Commonsense Knowledge Graph

Dynamic commonsense knowledge fused method for Chinese implicit sentiment analysis

Compared with explicit sentiment analysis that attracts considerable attention, implicit sentiment analysis is a more difficult task due to the lack of sentimental words. The abundant information in an external sentimental knowledge base can play a significant complementary and expansion role. In this paper, a sentimental commonsense knowledge graph embedded multi-polarity orthogonal attention model is proposed to learn the implication of the implicit sentiment. We analyzed the effectiveness of different knowledge relations in the ConceptNet knowledge base in detail, and proposed a matching and filtering method to distill useful knowledge tuples for implicit sentiment analysis automatically. By introducing the sentimental information in the knowledge base, the proposed model can extend the semantic of a sentence with an implicit sentiment. Then, a bi-directional long–short term memory model with multi-polarity orthogonal attention is adopted to fuse the distilled sentimental knowledge with the semantic embedding, effectively enriching the representation of sentences. Experiments on the SMP2019-ECISA implicit sentiment dataset show that our model fully utilizes the information of the knowledge base and improves the performance of Chinese implicit sentiment analysis.

CK-Encoder: Enhanced Language Representation for Sentence Similarity

In recent years, neural networks have been widely used in natural language processing, especially in sentence similarity modeling. Most of the previous studies focused on the current sentence, ignoring the commonsense knowledge related to the current sentence in the task of sentence similarity modeling. Commonsense knowledge can be remarkably useful for understanding the semantics of sentences. CK-Encoder, which can effectively acquire commonsense knowledge to improve the performance of sentence similarity modeling, is proposed in this paper. Specifically, the model first generates a commonsense knowledge graph of the input sentence and calculates this graph by using the graph convolution network. In addition, CKER, a framework combining CK-Encoder and sentence encoder, is introduced. Experiments on two sentence similarity tasks have demonstrated that CK-Encoder can effectively acquire commonsense knowledge to improve the capability of a model to understand sentences.

You should know more: Learning external knowledge for visual dialog

Visual dialog is a task that two agents complete a multi-round conversation based on an image, a caption, and dialog histories. Despite the recent progress, existing methods still undergo degradation on the condition of complex scenarios. Handling these scenarios depends on logical reasoning that requires commonsense priors. In this paper, we propose a novel visual dialog pipeline named Structured Knowledge-Aware Network (SKANet), consisting of an Image Knowledge-Aware Module and a Caption Knowledge-Aware Module. Specifically, the Image and Caption Knowledge-Aware Modules construct commonsense knowledge graphs from ConceptNet. We apply SKANet to two sub-tasks: the conventional visual dialog and a goal-oriented visual dialog named ‘image guessing’. For the conventional visual dialog, the SKANet is combined with an additional Multi-Modality Fusion Module, which is designed to explore the visual content and the textual context about the dialog history. For the goal-oriented visual dialog, we directly apply the Image and Caption Knowledge-Aware Modules to two agents, respectively. Experimental results on VisDial v0.9 and VisDial v1.0 datasets show that our proposed method effectively outperforms comparative methods on both sub-tasks.