Improving situational awareness with collective artificial intelligence over knowledge graphs

Abstract

Situational awareness (SA) was defined as perception of environmental elements within the situation, comprehension, and projection of future status. Here the perception is the representation of sensory information that includes multi-typed interacting objects forming knowledge graphs. A typical problem in artificial intelligence (AI) research is to learn representations of objects that preserve structural information in knowledge graphs (KGs). Existing methods assume an AI agent has a complete knowledge graph, and any kind of prediction can be made accurately by a single AI. However, the real world needs multiple AI agents (e.g., warfighters, citizens) to collectively make a prediction. Each AI has a different, incomplete view of the knowledge graph with noise. In this work, I present a novel approach to improve representation learning and thus to improve SA with collective AI over KGs. I present the approach in four parts. First, I introduce knowledge graph and its nature of heterogeneity with multiple examples in the real world. Second, I discuss four ideas of making prediction with collective AI: prediction ensemble, data aggregation, representation aggregation, and joint representation learning. Third, I describe two state-of-the-art models to learn object representations from heterogeneous graphs: one is path- based embedding and the other is a graph neural network (GNN). Lastly, I present a new GNN framework that jointly learns object representations from multiple agents. Experimental results demonstrate that collective AI performs significantly better than individual AI. In future work, I discuss about federated learning that may improve security and privacy of the framework, which is quite necessary when any type of sharing (e.g., raw data, object representations, or learning process) is sensitive.