Systematic Fuzz Testing Techniques on a Nanosatellite Flight Software for Agile Mission Development

Abstract

The success of CubeSat space missions depends on the ability to perform properly in a harsh environment. A key component in space missions is the flight software, which manages all of the processes executed by the satellite on its onboard computer. Literature shows that CubeSat missions suffer high infant mortality, and many spacecraft failures are related to flight software errors, some of them resulting in complete mission loss. Extensive operation testing is the primary technique used by CubeSats developers to ensure flight software quality and avoid such failures. The “New Space” requirements pressure to add “agility” to the software development, which could limit the capacity to test. While advanced and beneficial software testing techniques are found in the software engineering field, CubeSat software solutions mostly rely on unit testing, software in the loop simulation, and hardware in the loop simulation. In this work, fuzz testing techniques were developed, implemented, and evaluated as a manner to expedite operational testing of CubeSats while maintaining their completeness. The impact of the tools was evaluated by using the three new 3U CubeSats under development at the University of Chile. We identified twelve bugs not covered by classic testing strategies in less than three days. These failures were reported, fixed, and characterized by the developers in eight sprint sessions. Our results indicate that fuzz testing improved the completeness of flight software testing through automation and with almost no development interruption. Although our approach has been tested on the SUCHAI flight software, it applies to systems that follow a similar architecture.