An Investigation of Knowledge Gaps of Graduate Students Regarding Safety-Critical Systems Development: A Controlled Experiment

Abstract

<italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Contribution:</i> This article details the conduction of an experiment to investigate the knowledge that computer science graduate students have about safety-critical systems development, in particular, safety requirements specifications. Future research directions are also discussed. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Background:</i> Safety-critical systems have been increasingly used in many critical domains, requiring a rigorous development process as well as certification by regulatory agencies. However, it is not clear whether computer science courses deliver adequate education for students to develop such systems. Experimental research with these students could provide evidence on whether they have the minimum knowledge necessary to develop safety-critical systems. This type of research presents several advantages, such as reproducibility, high-level control, and specific conclusions, but has not been widely adopted to verify learning issues. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Research Question:</i> How to measure the understanding of students on safety-critical systems development? <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Methodology:</i> A controlled experiment was conducted with computer science graduate students in which they prepared safety requirements specifications. Three variables, i.e., understandability of the tasks, accuracy of the specifications, and effort required, were analyzed. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Findings:</i> This experiment made it possible to explicitly measure the effect of understandability on accuracy and effort. Our experimental scenario revealed that most graduate students do not know about safety engineering. Hence, it is paramount to rethink computer science curricula to include basic safety engineering concepts and how software-engineering-specific knowledge should be considered in the development of these systems. Besides, software architecture must receive special attention, as it is the backbone of all software-based systems, including safety-critical systems.