Commonsense Reasoning Research Articles

Self-Supervised Logic Induction for Explainable Fuzzy Temporal Commonsense Reasoning

Understanding temporal commonsense concepts, such as times of occurrence and durations is crucial for event-centric language understanding. Reasoning about such temporal concepts in a complex context requires reasoning over both the stated context and the world knowledge that underlines it. A recent study shows massive pre-trained LM still struggle with such temporal reasoning under complex contexts (e.g., dialog) because they only implicitly encode the relevant contexts and fail to explicitly uncover the underlying logical compositions for complex inference, thus may not be robust enough. In this work, we propose to augment LMs with the temporal logic induction ability, which frames the temporal reasoning by defining three modular components: temporal dependency inducer and temporal concept defuzzifier and logic validator. The former two components disentangle the explicit/implicit dependency between temporal concepts across context (before, after, ...) and the specific meaning of fuzzy temporal concepts, respectively, while the validator combines the intermediate reasoning clues for robust contextual reasoning about the temporal concepts. Extensive experimental results on TIMEDIAL, a challenging dataset for temporal reasoning over dialog, show that our method, Logic Induction Enhanced Contextualized TEmporal Reasoning (LECTER), can yield great improvements over the traditional language model for temporal reasoning.

Tackling Uncertainty Through Probabilistic Modelling of Proportionality in Military Operations

Just as every neuron in a biological neural network is a reinforcement learning agent, thus a component of a large and advanced structure is de facto a model, the two main components forming the principle of proportionality in military operations can be seen and are as a matter of fact two different entities and models. These are collateral damage depicting the unintentional effects affecting civilians and civilian objects, and military advantage symbolizing the intentional effects contributing to achieving the military objectives defined for military operation conducted. These two entities are complex processes relying on available information, projection on time to the moment of target engagement through estimation and are strongly dependent of common-sense reasoning and decision making. As a deduction, these two components and the proportionality decision result are processes surrounded by various sources and types of uncertainty. However, the existing academic and practitioner efforts in understanding the meaning, dimensions, and implications of the proportionality principle are considering military-legal and ethical lenses, and less technical ones. Accordingly, this research calls for a movement from the existing vision of interpreting proportionality in a possibilistic way to a probabilistic way. Henceforth, this research aims to build two probabilistic Machine Learning models based on Bayesian Belief Networks for assessing proportionality in military operations. The first model embeds a binary classification approach assessing if the engagement is proportional or disproportional, and the second model that extends this perspective based on previous research to perform multi-class classification for assessing degrees of proportionality. To accomplish this objective, this research follows the Design Science Research methodology and conducts an extensive literature for building and demonstrating the model proposed. Finally, this research intends to contribute to designing and developing explainable and responsible intelligent solutions that support human-based military targeting decision-making processes involved when building and conducting military operations.

Collaborative decision making is grounded in representations of other people's competence and effort.

By collaborating with others, humans can pool their limited knowledge, skills, and resources to achieve goals that outstrip the abilities of any one person. What cognitive capacities make human collaboration possible? Here, we propose that collaboration is grounded in an intuitive understanding of how others think and of what they can do-in other words, of their mental states and competence. We present a belief-desire-competence framework that formalizes this proposal by extending existing models of commonsense psychological reasoning. Our framework predicts that agents recursively reason how much effort they and their partner will allocate to a task, based on the rewards at stake and on their own and their collaborator's competence. Across three experiments (N = 249), we show that the belief-desire-competence framework captures human judgments in a variety of contexts that are critical to collaboration, including predicting whether a joint activity will succeed (Experiment 1), selecting incentives for collaborators (Experiment 2), and choosing which individuals to recruit for a collaborative task (Experiment 3). Our work provides a theoretical framework for understanding how commonsense psychological reasoning contributes to collaborative achievements. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

CEG: A joint model for causal commonsense events enhanced story ending generation.

With the success of pre-trained language models, the performance of story ending generation has been dramatically improved while remaining challenging due to the lack of commonsense reasoning ability. Most previous works mainly focus on using commonsense knowledge to enhance the implicit correlations between words but ignore the hidden causality of sentences or events. In this paper, we propose Causal commonsense Enhanced joint model for story ending Generation (CEG), which incorporates causal commonsense events knowledge to generate a reasonable story ending. Specifically, we first develop a commonsense events inference model trained on GLUCOSE, which converts static knowledge into a dynamic generation model to discover unseen knowledge. It uses prompts to produce various commonsense events behind the stories as pseudo-labels of the dataset. Then, we propose a joint model for the causal events inference task and the story ending generation task to inject inference knowledge into the generation, which consists of a shared encoder, an inference decoder, and a generation decoder. In the causal events inference task, we use the shared encoder and the inference decoder to reason the causal events behind each sentence of the story context to help the model better understand the story and provide long-distance dependencies for the story ending generation. In story ending generation, we combine the hidden states of the causal events with the story context to generate the story ending by the shared encoder and the generation decoder. We jointly train the model on two tasks so that the generation decoder produces the story endings that better match the clues. Experimental results on the ROCStories dataset show that our model outperforms the previous works, demonstrating the effectiveness of the joint model and the generated causal events.

An art painting style explainable classifier grounded on logical and commonsense reasoning

AbstractThis paper presents the art painting style explainable classifier named ANYXI. The classifier is based on art specialists’ knowledge of art styles and human-understandable color traits. ANYXI overcomes the principal flaws in the few art painting style classifiers in the literature. In this way, we first propose, using the art specialists’ studies, categorizations of the Baroque, Impressionism, and Post-Impressionism. Second, we carry out a human survey with the aim of validating the appropriateness of the color features used in the categorizations for human understanding. Then, we analyze and discuss the accuracy and interpretability of the ANYXI classifier. The study ends with an evaluation of the rationality of explanations automatically generated by ANYXI. We enrich the discussion and empirical validation of ANYXI by considering a quantitative and qualitative comparison versus other explainable classifiers. The reported results show how ANYXI is outstanding from the point of view of interpretability while keeping high accuracy (comparable to non-explainable classifiers). Moreover, automated generations are endowed with a good level of rationality.

Nonmonotonic inferences: Classical conclusions in an intuitionistic modal framework

Abstract We introduce a conceptual foundation for nonmonotonic reasoning which integrates an intuitionistic plurimodal logic with classical logic. The need for a multilogical system arises once we understand that to legitimately infer less than certain conclusions, we make two different kinds of assessments: an external one (what are the alternatives) and an internal one (which logic should govern the set of beliefs). We exploit this scheme to capture in novel ways basic nonmonotonic systems such as autoepistemic logic, default logic, one of the best known interpretations of negation-as-failure, as well as some conditional logics of normality. Among the aims of this paper is to explain why there are substantial overlaps between these systems and also to settle some problematic features that are of concern in logical investigations on common sense reasoning.

Deficits in Analytic and Common-Sense Reasoning in Schizophrenia.

Human rationality has a dual nature including analytic and common-sense thinking. Symptoms of schizophrenia have been suggested to be related to deficits in these aspects of logical reasoning. However, empirical studies investigating logical reasoning errors in schizophrenia and their clinical and neurocognitive correlates are scarce. Formal thought disorder and theory of mind (ToM) might be particularly important for understanding logical reasoning errors in schizophrenia. The current study compared the performances of 80 patients with schizophrenia with those of 49 healthy controls on syllogistic and counterfactual reasoning tasks and investigated clinical, neuropsychological, and social cognitive correlates of logical reasoning in schizophrenia. Patients with schizophrenia were impaired in both analytic and common-sense thinking. ToM impairment was a significant predictor of analytic reasoning abilities in schizophrenia. Executive functions and verbal memory were also significantly associated with analytic reasoning in schizophrenia. Further studies investigating logical reasoning errors in the early phases of the illness are needed.

Understanding Chinese Moral Stories with Further Pre-Training

The goal of moral understanding is to grasp the theoretical concepts embedded in a narrative by delving beyond the concrete occurrences and dynamic personas. Specifically, the narrative is compacted into a single statement without involving any characters within the original text, necessitating a more astute language model that can comprehend connotative morality and exhibit commonsense reasoning. The “pretraining + fine-tuning” paradigm is widely embraced in neural language models. In this paper, we propose an intermediary phase to establish an improved paradigm of “pre-training + further pre-training + finetuning”. Further pre-training generally refers to continual learning on task-specific or domain-relevant corpora before being applied to target tasks, which aims at bridging the gap in data distribution between the phases of pre-training and fine-tuning. Our work is based on a Chinese dataset named STORAL-ZH that composes of 4k human-written story-moral pairs. Furthermore, we design a two-step process of domain-adaptive pre-training in the intermediary phase. The first step depends on a newly-collected Chinese dataset of Confucian moral culture. And the second step bases on the Chinese version of a frequently-used commonsense knowledge graph (i.e. ATOMIC) to enrich the backbone model with inferential knowledge besides morality. By comparison with several advanced models including BERTbase, RoBERTa-base and T5-base, experimental results on two understanding tasks demonstrate the effectiveness of our proposed three-phase paradigm.

Web Perspectives in Robotics Applications

The realms of commonsense knowledge and reasoning, vehicle automation with full as well as partial autonomy, and human-robot collaboration, present growing areas of research in recent times, with much of the concerned data being disseminated through the Web and devices encompassing IoT (Internet of Things); the data per se being heterogeneous including plain text, images, audiovisuals, hypertext and hypermedia. Due to the advent of autonomous vehicles, there is a greater need for the embodiment of commonsense knowledge within their development in order to simulate subtle, intuitive aspects of human judgment. The field of robotics has often encountered collaborative tasks between humans and robots to enhance the respective activities involved and produce better results than humans or robots would achieve working by themselves. Accordingly, this article outlines and organizes some of the research occurring in these areas along with its Web perspectives and applications. Context related to human-robot collaboration and commonsense knowledge appears via a survey of the literature. Vehicle automation is significant with the relevant studies: its definition and methods of improvement are of focus in the article. Some work in this area makes an impact on smart manufacturing. There is discussion on how human-robot collaboration is beneficial, and how commonsense knowledge is useful for the collaboration to occur in an enhanced manner. This article would be potentially interesting to various communities, e.g. AI professionals, Web developers, robotics engineers, and data scientists.

Ontology-based hybrid commonsense reasoning framework for handling context abnormalities in uncertain and partially observable environments

Ambient intelligence (AmI) systems aim to provide users with context-aware assistance services intended to improve the quality of their lives in terms of autonomy, safety, and well-being. Taking the uncertainty and partial observability of these environments into account is of major importance for context recognition and, more specifically, to detect and solve context abnormalities such as those related to the user's behavior or those related to context attribute prediction. In this paper, an ontology-based framework integrating machine learning and probabilistic planning within commonsense reasoning is proposed to recognize the user's context and abnormalities associated with it. The reasoning is performed using event calculus in answer set programming (ECASP); ECASP allows for abductive and temporal reasoning, which results in an eXplainable AI (XAI) approach. A context ontology is proposed to axiomatize the reasoning and introduce the notion of probabilistic fluents into the EC formalism in order to perform probabilistic reasoning. The reasoning incorporates probabilistic planning based on a partially observable Markov decision process (POMDP) to solve knowledge incompleteness. To evaluate the proposed framework, real-life scenarios, based on the Orange4Home and SIMADL public datasets are implemented and discussed.

Lightweight dense video captioning with cross-modal attention and knowledge-enhanced unbiased scene graph.

Dense video captioning (DVC) aims at generating description for each scene in a video. Despite attractive progress for this task, previous works usually only concentrate on exploiting visual features while neglecting audio information in the video, resulting in inaccurate scene event location. In this article, we propose a novel DVC model named CMCR, which is mainly composed of a cross-modal processing (CM) module and a commonsense reasoning (CR) module. CM utilizes a cross-modal attention mechanism to encode data in different modalities. An event refactoring algorithm is proposed to deal with inaccurate event localization caused by overlapping events. Besides, a shared encoder is utilized to reduce model redundancy. CR optimizes the logic of generated captions with both heterogeneous prior knowledge and entities' association reasoning achieved by building a knowledge-enhanced unbiased scene graph. Extensive experiments are conducted on ActivityNet Captions dataset, the results demonstrate that our model achieves better performance than state-of-the-art methods. To better understand the performance achieved by CMCR, we also apply ablation experiments to analyze the contributions of different modules.

Assessment of Various Multimodal Fusion Approaches Using Synthetic Aperture Radar (SAR) and Electro-Optical (EO) Imagery for Vehicle Classification via Neural Networks.

Multimodal fusion approaches that combine data from dissimilar sensors can better exploit human-like reasoning and strategies for situational awareness. The performance of a six-layer convolutional neural network (CNN) and an 18-layer ResNet architecture are compared for a variety of fusion methods using synthetic aperture radar (SAR) and electro-optical (EO) imagery to classify military targets. The dataset used is the Synthetic and Measured Paired Labeled Experiment (SAMPLE) dataset, using both original measured SAR data and synthetic EO data. We compare the classification performance of both networks using the data modalities individually, feature level fusion, decision level fusion, and using a novel fusion method based on the three RGB-input channels of a residual neural network (ResNet). In the proposed input channel fusion method, the SAR and the EO imagery are separately fed to each of the three input channels, while the third channel is fed a zero vector. It is found that the input channel fusion method using ResNet was able to consistently perform to a higher classification accuracy in every equivalent scenario.

An experimental study measuring the generalization of fine‐tuned language representation models across commonsense reasoning benchmarks

AbstractIn the last 5 years, language representation models, such as BERT and GPT‐3, based on transformer neural networks, have led to enormous progress in natural language processing (NLP). One such NLP task is commonsense reasoning, where performance is usually evaluated through multiple‐choice question answering benchmarks. Till date, many such benchmarks have been proposed, and ‘leaderboards’ tracking state‐of‐the‐art performance on those benchmarks suggest that transformer‐based models are approaching human‐like performance. Because these are commonsense benchmarks, however, such a model should be expected to generalize, that is, at least in aggregate, should not exhibit excessive performance loss across independent commonsense benchmarks regardless of the specific benchmark on (the training set of) which it has been fine‐tuned. In this article, we evaluate this expectation by proposing a methodology and experimental study to measure the generalization ability of language representation models using a rigorous and intuitive metric. Using five established commonsense reasoning benchmarks, our experimental study shows that the models do not generalize well, and may be (potentially) susceptible to issues such as dataset bias. The results therefore suggest that current performance on benchmarks may be an over‐estimate, especially if we want to use such models on novel commonsense problems for which a ‘training’ dataset may not be available, for the language representation model, to fine‐tune on.

Boosting Generic Visual-Linguistic Representation With Dynamic Contexts

Pretraining large models on generous multi-modal corpora has accelerated the development of visual-linguistic (VL) representation and achieved great success on various vision-and-language downstream tasks. Learning these models is usually executed by predicting the randomly masked words of captions or patches in images. Such approaches, nevertheless, seldom explore the supervision of causalities behind the caption descriptions or the procedure of generating events beyond still images. In this work, we endow the pretrained models with high-level cognition by delving into dynamic contexts to model the visual and linguistic causalities uniformly. Specifically, we format the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">dynamic contexts</i> of an image as the sentences describing the events <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">before</i> , <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">on</i> , and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">after</i> image. Unlike traditional caption-wise similarity, we propose a novel dynamic contexts-based similarity (DCS) metric, in which the correlation of potential causes and effects besides immediate visual content are considered to measure the relevance among images. DCS can be further simplified by parameterizing event continuity to relax the requirements on dense contextual event annotations. A new pre-task is designed to minimize the feature distances of dynamically contextual relevant images and incorporate the event causality and commonsense knowledge into the VL representation learning. Models based on our dynamic contexts significantly outperform typical VL models on multiple cross-modal downstream tasks, including the conventional visual commonsense reasoning (VCR), visual question answering (VQA), zero-shot image-text retrieval, and extended image / event ordering tasks.

Detecting Malware Activities With MalpMiner: A Dynamic Analysis Approach

Day by day, malware as a service becomes more popular and easy to acquire, thus allowing anyone to start an attack without any technical background. In addition, advanced malware attacks get more sophisticated such as Advanced Persistent Threats (APTs). Such reasons introduce challenges for detecting such attacks. One of those challenges is the detection of malware activities early in order to prevent harm as much as possible. Therefore, this paper presents a trusted dynamic analysis approach based on Answer Set Programming (ASP), a logic engine inference named Malware-Logic-Miner (MalpMiner). ASP is a nonmonotonic reasoning engine built on an open-world assumption. Thus, it allows MalpMiner to adopt commonsense reasoning when capturing malware activities of any given binary. Furthermore, MalpMiner requires no prior training; therefore, it can scale up quickly to include more malware-attack attributes. Besides that, it considers the invoked application programming interfaces’ values, resulting in correct malware behaviour modelling. The baseline experiments prove the correctness of MalpMiner related to recognizing malware activities. Moreover, MalpMiner achieved a detection ratio of 99% with a false-positive rate of less than 1% while maintaining low computational costs and explaining the detection decision.

ASSESSING THE DEGREE OF INFLUENCE BETWEEN THE PARAMETERS OF THE STRUCTURAL-CAUSAL MODEL

Cause-and-effect relationships underpin modern science and the ability to teach computers human-like reasoning. The study of causality combined with modern advances in machine learning can greatly accelerate scientific progress. Modern scientific advancements emphasize the importance of not only establishing causality, but also assessing the degree of influence between cause-and-effect relationships. The work poses an interventionist and counterfactual question: «How will the values of the model parameters change due to the influence of the studied factor?» and «Which factor needs to be influenced for the fastest change in the value of the studied model parameter?». Formulating more precise hypotheses and accelerating their testing can be achieved by answering these questions. This article proposes a method for estimating the degree of influence between the parameters of the structural-causal model, which solves the problem of identifying the most influential factors, and also improves the study of causal relationships between parameters, which will allow improving the prediction and management of the model under study. The proposed approach of presenting data in a structural-causal model allows you to draw conclusions about what is the cause and what is the effect and take into account the influence of both direct and indirect cause-and-effect relationships. The assessment of the degree of influence determines the most significant parameters and the least influential factors. Consequently, predicting changes in model parameter values becomes more predictable and controllable. Conditional ignoring of the least influential parameters can be used in the model optimization process.

EREC: Enhanced Language Representations with Event Chains

The natural language model BERT uses a large-scale unsupervised corpus to accumulate rich linguistic knowledge during its pretraining stage, and then, the information is fine-tuned for specific downstream tasks, which greatly improves the understanding capability of various natural language tasks. For some specific tasks, the capability of the model can be enhanced by introducing external knowledge. In fact, these methods, such as ERNIE, have been proposed for integrating knowledge graphs into BERT models, which significantly enhanced its capabilities in related tasks such as entity recognition. However, for two types of tasks, commonsense causal reasoning and predicting the ending of stories, few previous studies have combined model modification and process optimization to integrate external knowledge. Therefore, referring to ERNIE, in this paper, we propose enhanced language representation with event chains (EREC), which focuses on keywords in the text corpus and their implied relations. Event chains are integrated into EREC as external knowledge. Furthermore, various graph networks are used to generate embeddings and to associate keywords in the corpus. Finally, via multi-task training, external knowledge is integrated into the model generated in the pretraining stage so as to enhance the effect of the model in downstream tasks. The experimental process of the EREC model is carried out with a three-stage design, and the experimental results show that EREC has a deeper understanding of the causal relationship and event relationship contained in the text by integrating the event chains, and it achieved significant improvements on two specific tasks.

Winograd schemata and other datasets for anaphora resolution in Hungarian

AbstractThe Winograd Schema Challenge (WSC, proposed by Levesque, Davis &amp; Morgenstern 2012) is considered to be the novel Turing Test to examine machine intelligence. Winograd schema questions require the resolution of anaphora with the help of world knowledge and commonsense reasoning. Anaphora resolution is itself an important and difficult issue in natural language processing, therefore, many other datasets have been created to address this issue. In this paper we look into the Winograd schemata and other Winograd-like datasets and the translations of the schemata to other languages, such as Chinese, French and Portuguese. We present the Hungarian translation of the original Winograd schemata and a parallel corpus of all the translations of the schemata currently available. We also adapted some other anaphora resolution datasets to Hungarian. We aim to discuss the challenges we faced during the translation/adaption process.

Integration of ontology reasoning-based monocular cues in deep learning modeling for single image depth estimation in urban driving scenarios

Humans are able to estimate the depth of objects in their environment even using only one eye through the use of what are known as monocular cues. In this paper, we aim to integrate human knowledge and human-like reasoning used for monocular depth estimation within deep neural networks. The idea is to support the network in order to help it learn in an explicit and fast way the essential cues for the target task. For this purpose, we investigate the possibility of directly integrating geometric, semantic, and contextual information into the monocular depth estimation process. We propose exploiting an ontology model in a deep learning context to represent the urban environment as a structured set of concepts linked with semantic relationships. Monocular cues information are extracted through reasoning performed on the proposed ontology and are fed together with the RGB image in a multistream way into the deep neural network for depth estimation. Our approach is validated and evaluated on widespread benchmark datasets: KITTI, CityScapes, and AppolloScape. The obtained results show that the proposed method improves upon the state-of-the-art monocular depth estimation deep models and shows promising results regarding cross-evaluation, mainly for unseen driving scenarios.

Survey on the Research Progress and Development Trend of Vision-and-Language Navigation

Vision-and-language navigation is a newly emerging research topic developing rapidly in recent years, and it is one of the representative research tasks in the frontier field of vision-language interaction. The goal of this task is to realize autonomous navigation based on visual perception of environment according to language instructions given by human. This paper reviews the recent progress in vision-and-language navigation. Firstly, the research content of this task is introduced, and the three main problems and challenges of cross-modal semantic alignments, semantic understanding and reasoning, and generalization ability enhancement are analyzed. Secondly, commonly-used datasets and evaluation metrics are listed. Thirdly, the research progress of this task is summarized from four aspects of imitation learning, reinforcement learning, self-supervised learning and other methods, and the effects of the typical solutions are carefully compared and analyzed. Fourthly, the current research trends of this task are discussed, which mainly include continuous environment navigation, advanced complex instruction comprehension and common sense reasoning. Finally, the future development directions such as 3D visual-and-language navigation, embodied question answering and interactive question answering are further discussed and prospected.