Does The Complex SoS Have Negative Emergent Behavior? Looking For Violations Formally

Abstract

A complex system is characterized by emergence of global properties. These emergent properties are very difficult to anticipate just from complete knowledge of component behaviors. Characteristics of complex systems discussed in literature include emergence, hierarchical organization and numerosity. Recently, there has been an increase on the adoption of various neural network-based machine learning models to govern the functionality and behavior of systems. With this increasing system complexity, there is increasing challenge in attaining confidence in systems. The ease with which systems are getting interconnected is permeating numerous system-of-systems (SoS), wherein multiple independent systems are expected to interact and collaborate to achieve unparalleled levels of functionality. Traditional verification and validation approaches are often inadequate to bring in the nuances of potential emergent behavior in a system-of-system, which may be positive or negative. In this paper, we look for violations formally in the emergent behavior of a complex SoS. The case study pertains to a swarm of autonomous UAVs flying in a formation, and dynamically changing the shape of the formation, to support varying mission scenarios. We use a tool called CBMC which is a bounded model checker that looks at properties in a small defined region and bound and argues on its correctness. The effectiveness and performance of the approach are quantified.