Reasoning Module Research Articles

KNOWLEDGE REPRESENTATION AND REASONING

This paper provides a professional approach to construct a Knowledge Representation and Reasoning module. The development of AGI agents requires architecture modeled after human cognition and this also provides a framework for agents that have interaction with the actual world and to constitute and use it for making decisions that capture and permit implementation of a behavior.

A Deep Detection Network Based on Interaction of Instance Segmentation and Object Detection for SAR Images

Ship detection is a challenging task for synthetic aperture radar (SAR) images. Ships have arbitrary directionality and multiple scales in SAR images. Furthermore, there is a lot of clutter near the ships. Traditional detection algorithms are not robust to these situations and easily cause redundancy in the detection area. With the continuous improvement in resolution, the traditional algorithms cannot achieve high-precision ship detection in SAR images. An increasing number of deep learning algorithms have been applied to SAR ship detection. In this study, a new ship detection network, known as the instance segmentation assisted ship detection network (ISASDNet), is presented. ISASDNet is a two-stage detection network with two branches. A branch is called an object branch and can extract object-level information to obtain positioning bounding boxes and classification results. Another branch called the pixel branch can be utilized for instance segmentation. In the pixel branch, the designed global relational inference layer maps the features to interaction space to learn the relationship between ship and background. The global reasoning module (GRM) based on global relational inference layers can better extract the instance segmentation results of ships. A mask assisted ship detection module (MASDM) is behind the two branches. The MASDM can improve detection results by interacting with the outputs of the two branches. In addition, a strategy is designed to extract the mask of SAR ships, which enables ISASDNet to perform object detection training and instance segmentation training at the same time. Experiments carried out two different datasets demonstrated the superiority of ISASDNet over other networks.

Reinforcement Learning for Spatial Reasoning in Strategy Games

One of the major weaknesses of current real-time strategy (RTS) game agents is handling spatial reasoning at a high level. One challenge in developing spatial reasoning modules for RTS agents is to evaluate the ability of a given agent for this competency due to the inevitable confounding factors created by the complexity of these agents. We propose a simplified game that mimics spatial reasoning aspects of more complex games, while removing other complexities. Within this framework, we analyze the effectiveness of classical reinforcement learning for spatial management in order to build a detailed evaluative standard across a broad set of opponent strategies. We show that against a suite of opponents with fixed strategies, basic Q-learning is able to learn strategies to beat each. In addition, we demonstrate that performance against unseen strategies improves with prior training from other distinct strategies. We also test a modification of the basic algorithm to include multiple actors, to speed learning and increase scalability. Finally, we discuss the potential for knowledge transfer to more complex games with similar components.

The business transformation enterprise architecture framework for innovation The role of artificial intelligence in the global business education (RAIGBE)

This article analyses the role of Global Business Education (GBE) and proposes the Applied Holistic Mathematical Model for GBE (AHMM4GBE). The AHMM4GBE is based on a lifetime long research on business transformations, Artificial Intelligence (AI), applied mathematics, software modelling, business engineering, educational systems, financial analysis, security and enterprise architecture. The used research methodology is based on the author’s authentic mixed research method that is supported by a mainly qualitative reasoning module. AHMM4GBE’s formalism mimics the human brain, by using empirical processes that are mainly based on heuristics. The AHMM4GBE is used to implement a decision-making system (or an expert system) to support a GBE and uses a behaviour-driven development environment that can be easily adopted by any organization. The development environment can be used by any team member without any prior computer sciences qualifications. The AHMM4GBE is used to estimate the Role of AI in GEB’s (RAIGBE) context and tries to estimate the roles of the giants in this domain, like USA, China, and India; and what would be the real role of the European Union and France. The uniqueness of this research is that the AHMM4GBE promotes a holistic unbundling process, the alignment of transformation strategies to support GBE’s evolution. For a successful integration of the AHMM4GBE in projects, the manager’s profile and role are crucial, where his decisions are supported by the selection, implementation and processing of critical success factors. A holistic systemic system approach is the optimal choice to integrate an RAI4GBE.

Similarity Reasoning and Filtration for Image-Text Matching

Image-text matching plays a critical role in bridging the vision and language, and great progress has been made by exploiting the global alignment between image and sentence, or local alignments between regions and words. However, how to make the most of these alignments to infer more accurate matching scores is still underexplored. In this paper, we propose a novel Similarity Graph Reasoning and Attention Filtration (SGRAF) network for image-text matching. Specifically, the vector-based similarity representations are firstly learned to characterize the local and global alignments in a more comprehensive manner, and then the Similarity Graph Reasoning (SGR) module relying on one graph convolutional neural network is introduced to infer relation-aware similarities with both the local and global alignments. The Similarity Attention Filtration (SAF) module is further developed to integrate these alignments effectively by selectively attending on the significant and representative alignments and meanwhile casting aside the interferences of non-meaningful alignments. We demonstrate the superiority of the proposed method with achieving state-of-the-art performances on the Flickr30K and MSCOCO datasets, and the good interpretability of SGR and SAF with extensive qualitative experiments and analyses.

A Graph Reasoning Network for Multi-turn Response Selection via Customized Pre-training

We investigate response selection for multi-turn conversation in retrieval-based chatbots. Existing studies pay more attention to the matching between utterances and responses by calculating the matching score based on learned features, leading to insufficient model reasoning ability. In this paper, we propose a graph- reasoning network (GRN) to address the problem. GRN first conducts pre-training based on ALBERT using next utterance prediction and utterance order prediction tasks specifically devised for response selection. These two customized pre-training tasks can endow our model with the ability of capturing semantical and chronological dependency between utterances. We then fine-tune the model on an integrated network with sequence reasoning and graph reasoning structures. The sequence reasoning module conducts inference based on the highly summarized context vector of utterance-response pairs from the global perspective. The graph reasoning module conducts the reasoning on the utterance-level graph neural network from the local perspective. Experiments on two conversational reasoning datasets show that our model can dramatically outperform the strong baseline methods and can achieve performance which is close to human-level.

Temporal Relational Modeling with Self-Supervision for Action Segmentation

Temporal relational modeling in video is essential for human action understanding, such as action recognition and action segmentation. Although Graph Convolution Networks (GCNs) have shown promising advantages in relation reasoning on many tasks, it is still a challenge to apply graph convolution networks on long video sequences effectively. The main reason is that large number of nodes (i.e., video frames) makes GCNs hard to capture and model temporal relations in videos. To tackle this problem, in this paper, we introduce an effective GCN module, Dilated Temporal Graph Reasoning Module (DTGRM), designed to model temporal relations and dependencies between video frames at various time spans. In particular, we capture and model temporal relations via constructing multi-level dilated temporal graphs where the nodes represent frames from different moments in video. Moreover, to enhance temporal reasoning ability of the proposed model, an auxiliary self-supervised task is proposed to encourage the dilated temporal graph reasoning module to find and correct wrong temporal relations in videos. Our DTGRM model outperforms state-of-the-art action segmentation models on three challenging datasets: 50Salads, Georgia Tech Egocentric Activities (GTEA), and the Breakfast dataset. The code is available at https://github.com/redwang/DTGRM.

A knowledge based system for the management of a time stamped uncertain observation set with application on preserving mobility

The aim of this study is to maintain up-to-date information about the current state of elderly people that are medically followed for risks of fall. Our proposal consists of an individual information database management system that can provide information on-demand on various variables. Such a system has to deal with several sources of uncertainty: lack of information, evolving information and reliability of the information sources. We consider that the features of the person may evolve with time causing uncertainty due to obsolete information. Our context includes new information received bit by bit, with no possibility to collect all required information at once. This paper establishes a first proposal to manage a set of uncertain observations, in order to reduce erroneous and obsolete information while keeping the benefit of previously collected information. We propose an architecture of the system based on a probabilistic knowledge model about the characteristics of interest, a set of decay functions that help to evaluate the confidence degree in previous observations, and a reasoning module to manage new observations, maintain the compatibility and the quality of the observation set. We detail the algorithms of the reasoning module, and the algorithm to update the confidence degree of the observations.

Multi-scale GCN-assisted two-stage network for joint segmentation of retinal layers and discs in peripapillary OCT images.

An accurate and automated tissue segmentation algorithm for retinal optical coherence tomography (OCT) images is crucial for the diagnosis of glaucoma. However, due to the presence of the optic disc, the anatomical structure of the peripapillary region of the retina is complicated and is challenging for segmentation. To address this issue, we develop a novel graph convolutional network (GCN)-assisted two-stage framework to simultaneously label the nine retinal layers and the optic disc. Specifically, a multi-scale global reasoning module is inserted between the encoder and decoder of a U-shape neural network to exploit anatomical prior knowledge and perform spatial reasoning. We conduct experiments on human peripapillary retinal OCT images. We also provide public access to the collected dataset, which might contribute to the research in the field of biomedical image processing. The Dice score of the proposed segmentation network is 0.820 ± 0.001 and the pixel accuracy is 0.830 ± 0.002, both of which outperform those from other state-of-the-art techniques.

Relation-aware Siamese region proposal network for visual object tracking

The backbone networks used in Siamese trackers are relatively shallow, such as AlexNet and VGGNet, resulting in insufficient features for tracking task. Therefore, this paper focuses on extracting more discriminative features to improve the performance of Siamese trackers. By comprehensive experimental validations, this goal is achieved through a simple yet effective framework referred as relation-aware Siamese region proposal network (Ra-SiamRPN). Firstly, the deep backbone network ResNet-50 is adopted to extract both low-level detail features and high-level semantic features of an image. Then we propose the feature fusion module (FFM), which can combine low-level detail features with high-level semantic features effectively. Furthermore, we propose the relation reasoning module (RRM) to perform the global relation reasoning in multiple disjoint regions. RRM can generate discriminative information to enhance the features generated by ResNet-50. Extensive experiments are conducted on the dataset OTB2015, VOT2016, VOT2018, UAV123 and LaSOT. The experiment results indicate that Ra-SiamRPN achieves competitive performance with the current advanced algorithms and shows good real-time performance. To be highlighted, in the experiments conducted on the large-scale dataset LaSOT, the success score and the normalized precision score of Ra-SiamRPN are 0.495 and 0.576, respectively. These performance indexes are better than the second best tracker MDNet 24.7% and 25.2%.

Realize your surroundings: Exploiting context information for small object detection

Small object detection is a highly challenging problem due to the limited resolution and information of small objects. Current state-of-the-art detectors only utilize the appearance feature to locate and classify objects. However, such detectors are prone to failure when detecting small objects, especially in the case of heavy appearance changes and background distractors, in which the appearance feature alone is not sufficient for robust detection. Exploiting context information in the surrounding scene can be highly beneficial in such cases. In this paper, we propose a novel detector, the Internal-External Network (IENet), which uses both the appearance and context information of the object for robust detection. In the proposed approach, small object detection is improved from feature extraction, proposal location, and classification. Specifically, three customized modules are designed, including the Bidirectional Feature Fusion Module (Bi-FFM), Context Reasoning Module (CRM), and Context Feature Augmentation Module (CFAM). Bi-FFM is designed to capture the internal feature of objects by transferring the semantic feature of deeper-level layers to lower-level layers and the detailed feature of lower-level layers to deeper-level layers in neural networks. The proposed approach not only utilizes the hierarchy of convolutional features but also improves its prediction via context relationships. CRM is designed to improve the quality of region proposals by context reasoning that uses easily detected objects to help understand hard ones. Furthermore, CFAM is designed to learn pair-wise relations between region proposals produced by CRM, and such relations are used to produce global feature information associated with the region proposals for accurate classification. Extensive experiments are conducted on the challenging COCO and WIDER FACE datasets to demonstrate the effectiveness of the proposed approach. Experimental results show that the detection performance of small objects is greatly improved over state-of-the-art detectors.

CogKR: Cognitive Graph for Multi-hop Knowledge Reasoning

Inferring new facts from an existing knowledge graph with explainable reasoning processes is an important problem, known as knowledge graph (KG) reasoning. The problem is often formulated as finding the specific path that represents the query relation and connects the query entity and the correct answer. However, due to the limited expressiveness of individual paths, the majority of previous works failed to capture the complex subgraph structure in the graph. We propose CogKR that traverses the knowledge graph to conduct multi-hop reasoning. More specifically, motivated by the dual process theory from cognitive science, our framework is composed of an extension module and a reasoning module. By setting up a cognitive graph through iteratively coordinating the two modules, CogKR can cope with more complex reasoning scenarios in the form of subgraphs instead of individual paths. Experiments on three knowledge graph reasoning benchmarks demonstrate that CogKR achieves significant improvements in accuracy compared with previous methods while providing the explainable capacity. Moreover, we evaluate CogKR on the challenging one-shot link prediction task, exhibiting the superiority of the framework on accuracy and scalability compared to the state-of-the-art approaches.

Non-Homogeneous Haze Removal via Artificial Scene Prior and Bidimensional Graph Reasoning.

Due to the lack of natural scene and haze prior information, it is greatly challenging to completely remove the haze from a single image without distorting its visual content. Fortunately, the real-world haze usually presents non-homogeneous distribution, which provides us with many valuable clues in partial well-preserved regions. In this paper, we propose a Non-Homogeneous Haze Removal Network (NHRN) via artificial scene prior and bidimensional graph reasoning. Firstly, we employ the gamma correction iteratively to simulate artificial multiple shots under different exposure conditions, whose haze degrees are different and enrich the underlying scene prior. Secondly, beyond utilizing the local neighboring relationship, we build a bidimensional graph reasoning module to conduct non-local filtering in the spatial and channel dimensions of feature maps, which models their long-range dependency and propagates the natural scene prior between the well-preserved nodes and the nodes contaminated by haze. To the best of our knowledge, this is the first exploration to remove non-homogeneous haze via the graph reasoning based framework. We evaluate our method on different benchmark datasets. The results demonstrate that our method achieves superior performance over many state-of-the-art algorithms for both the single image dehazing and hazy image understanding tasks. The source code of the proposed NHRN is available on https://github.com/whrws/NHRNet.

Person Retrieval in Surveillance Videos Via Deep Attribute Mining and Reasoning

Person retrieval largely relies on the appearance features of pedestrians. This task is rather more difficult in surveillance videos due to the limitations of extracting robust appearance features brought by the cross-view and cross-camera data with lower image resolution, motion blur, occlusion and other kinds of image degradation. To build up a more reliable person retrieval system, recent works introduced appearance attribute models to describe and distinguish different persons with high-level semantic concepts. Despite the progress of previous works, the value of utilizing appearance attributes is still under-explored. On one hand, existing methods lack for concise and precise attribute representations that are specific for each attribute category and, in the meantime, are able to filter noisy information in irrelevant spatial locations and useless patterns. On the other hand, correlation and reasoning between different attributes are neglected, which could generate more useful information and add more robustness to the retrieval system. In this paper, we propose an Attribute Mining and Reasoning (AMR) framework which is capable to handle the issues in question. The AMR makes better use of appearance attributes with two main components. First, the AMR disentangles the representations of different attributes by localizing their spatial positions and identifying their effective patterns in a weakly supervised manner. To achieve more reliable localization, we propose the Attribute Localization Ensemble (ALE) module that is consisted of multiple localization heads and a voting mechanism. Second, we introduce the Attribute Reasoning (AR) module to correlate different attributes together with the global appearance features and discover their latent relations to generate more comprehensive descriptions of pedestrians. Extensive experiments on DukeMTMC-ReID and Market-1501 datasets demonstrate the effectiveness of the proposed AMR framework as well as its superiority over the existing state-of-the-art methods. The AMR model also shows great generalization ability on the unseen CUHK03 dataset when it is only trained on Market-1501 dataset.

A novel reasoning mechanism for multi-label text classification

The aim in multi-label text classification is to assign a set of labels to a given document. Previous classifier-chain and sequence-to-sequence models have been shown to have a powerful ability to capture label correlations. However, they rely heavily on the label order, while labels in multi-label data are essentially an unordered set. The performance of these approaches is therefore highly variable depending on the order in which the labels are arranged. To avoid being dependent on label order, we design a reasoning-based algorithm named Multi-Label Reasoner (ML-Reasoner) for multi-label classification. ML-Reasoner employs a binary classifier to predict all labels simultaneously and applies a novel iterative reasoning mechanism to effectively utilize the inter-label information, where each instance of reasoning takes the previously predicted likelihoods for all labels as additional input. This approach is able to utilize information between labels, while avoiding the issue of label-order sensitivity. Extensive experiments demonstrate that our method outperforms state-of-the art approaches on the challenging AAPD dataset. We also apply our reasoning module to a variety of strong neural-based base models and show that it is able to boost performance significantly in each case.

Cloud-Based Industrial Cyber–Physical System for Data-Driven Reasoning: A Review and Use Case on an Industry 4.0 Pilot Line

Nowadays, reconfiguration and adaptation by means of optimal re-parameterization in Industrial Cyber–Physical Systems (ICPSs) is one of the bottlenecks for the digital transformation of the manufacturing industry. This article proposes a cloud-to-edge-based ICPS equipped with machine learning techniques. The proposed reasoning module includes a learning procedure based on two reinforcement learning techniques, running in parallel, for updating both the data-conditioning and processing strategy and the prediction model. The presented solution distributes computational resources and analytic engines in multiple layers and independent modules, increasing the smartness and the autonomy for monitoring and control the behavior at the shop floor level. The suitability of the proposed solution, evaluated in a pilot line, is endorsed by fast time response (i.e., 0.01 s at the edge level) and the appropriate setting of optimal operational parameters for guaranteeing the desired quality surface roughness during macro- and micro-milling operations.

Design of Fast Fault Diagnosis System for Transformer Equipment based on CBR and RBR

After studying the common defects and faults of transformers, we design and propose a fast fault diagnosis system for transformer equipment based on CBR(Case-Based Reasoning) and RBR(Rule-Based Reasoning). In the system, test cases are firstly matched with the cases. If there are similar cases, CBR is used to obtain the diagnosis conclusion. If there are no similar cases, the rule reasoning module based on FTA (Fault Tree Analysis) is launched. This module subdivides transformer fault modes into winding, core, bushing, on-load tap changer, cooling system, non-electric protection and other fault modes, and forms rules. Based on this situation, a transformer equipment fault tree is established to serve the rapid diagnosis system. The structural characteristics and fault modes of transformer windings, iron cores, bushings, on-load tap changers, cooling systems, non-electrical protection and other equipment are subdivided. And then, a fast fault diagnosis model and system for transformer equipment are established. Finally, the case study proved that the system can diagnose and analyse the equipment faults based on the comprehensive multi-source data of transformers, and the diagnosis conclusion is reliable and practical, which has provided support for realizing the state maintenance of transformers.

Effect of communication skills training on radiation therapy student's confidence and interactions during their first clinical placement

IntroductionThe University of Newcastle, Radiation Therapy degree (RT) includes a clinical reasoning module that provides students with the opportunity to learn and practice communication skills before their first clinical placement. This study investigates students' perceptions of how well the module prepared them for placement and explores students’ perceptions of their communication experiences during placement. MethodsA single survey was administered to students four months following placement (six months after delivery of module), and included a request for perceptions pre-module, post-module and pre and post placement via an online survey comprising 46 closed, Likert scale and open-response questions. Assessment of changes in student confidence, the value of preparation for placement and perceptions concerning interpersonal skills with patients and radiation therapists (RTTs) were examined. Survey data was analysed quantitatively (counts, Fisher's exact) assessing statistical differences in confidence and Weighted Sum Average (WSA) of Likert scales. Qualitative findings were supported by written comments. ResultsParticipants reported increasing confidence levels between pre-module (WSA - 1.13), and post-module (2.88), which was statistically significant (p = 0.039) and between pre-placement (2.0) and after placement (3.38), in their ability to communicate with patients. Similar changes in confidence were noted with interactions with RTTs (WSA scores 1.0, 2.13, 1.75, 3.25 resp). Students reported communication strategies prepared them well for patient interactions. Positive reinforcement from RTTs was acknowledged as creating immediate support of their current patient skills. ConclusionUndertaking the clinical reasoning module successfully increased students’ confidence and provided opportunities to develop appropriate communication skills for placement. Implications for practiceIncreased levels of confidence and positive reinforcement from RTTs should positively impact students’ knowledge and skills, and provide introspection for improvement for future placements.

Cross-modal learning with prior visual relation knowledge

Visual relational reasoning is a central component in recent cross-modal analysis tasks, which aims at reasoning about the visual relationships between objects and their properties. These relationships provide rich semantics and help to enhance the visual representation for improving cross-modal learning. Previous works have succeeded in modeling latent visual relationships or rigid-categorized visual relationships. However, these kinds of methods leave out the problem of ambiguity inherent in the visual relationships because of the diverse relational semantics of different visual appearances. In this work, we explore to model the visual relationships by context-aware representations based on human prior knowledge. Based on such representations, we novelly propose a plug-and-play visual relational reasoning module to enhance image encoding. Specifically, we design an Anisotropic Graph Convolution to utilize the information of relation embeddings and relation directionality between objects for generating relation-aware image representations. We demonstrate the effectiveness of the relational reasoning module by applying it to both Visual Question Answering (VQA) and Cross-Modal Information Retrieval (CMIR) tasks. Extensive experiments are conducted on VQA 2.0 and CMPlaces datasets and superior performance is reported when comparing with state-of-the-art works.

Learning spatial-temporal deformable networks for unconstrained face alignment and tracking in videos

In this paper, we propose a spatial-temporal deformable networks approach to investigate both problems of face alignment in static images and face tracking in videos under unconstrained environments. Unlike conventional feature extractions which cannot explicitly exploit augmented spatial geometry for various facial shapes, in our approach, we propose a deformable hourglass networks (DHGN) method, which aims to learn a deformable mask to reduce the variances of facial deformation and extract attentional facial regions for robust feature representation. However, our DHGN is limited to extract only spatial appearance features from static facial images, which cannot explicitly exploit the temporal consistency information across consecutive frames in videos. For efficient temporal modeling, we further extend our DHGN to a temporal DHGN (T-DHGN) paradigm particularly for video-based face alignment. To this end, our T-DHGN principally incorporates with a temporal relational reasoning module, so that the temporal order relationship among frames is encoded in the relational feature. By doing this, our T-DHGN reasons about the temporal offsets to select a subset of discriminative frames over time steps, thus allowing temporal consistency information memorized to flow across frames for stable landmark tracking in videos. Compared with most state-of-the-art methods, our approach achieves superior performance on folds of widely-evaluated benchmarking datasets. Code will be made publicly available upon publication.