Model-based testing of NASA’s GMSEC, a reusable framework for ground system software

Abstract

We present an empirical study in which model-based testing (MBT) was applied to the software bus of NASA’s Goddard Mission Service Evolution Center (GMSEC), a reusable software framework. The goal was to study the feasibility of using MBT on a real-world software system that was designed to be flexible. GMSEC has three levels of flexibility: 1) loose application coupling through a software bus based on the publish–subscribe architectural style, 2) language independence by providing APIs to the bus in several programming languages, 3) middleware independence by providing wrappers for several middlewares that are supported by the software bus. The novelty brought forward in this paper is that one model and one set of generated test cases were used as the basis to test the software bus for behavioral consistency across multiple programming languages and middleware wrappers. The comparison of costs and benefits from using finite state machines (FSM) vs. extended FSMs (EFSM) when used for MBT on a real-world system is also novel. The case study shows that it was feasible, even for a programmer who neither knew MBT nor the system under test, to successfully apply MBT to a flexible system such as GMSEC and that MBT could within reasonable effort detect non-trivial defects in a fielded system.