Critical Software Development Research Articles

An empirical investigation of challenges of specifying training data and runtime monitors for critical software with machine learning and their relation to architectural decisions

The development and operation of critical software that contains machine learning (ML) models requires diligence and established processes. Especially the training data used during the development of ML models have major influences on the later behaviour of the system. Runtime monitors are used to provide guarantees for that behaviour. Runtime monitors for example check that the data at runtime is compatible with the data used to train the model. In a first step towards identifying challenges when specifying requirements for training data and runtime monitors, we conducted and thematically analysed ten interviews with practitioners who develop ML models for critical applications in the automotive industry. We identified 17 themes describing the challenges and classified them in six challenge groups. In a second step, we found interconnection between the challenge themes through an additional semantic analysis of the interviews. We explored how the identified challenge themes and their interconnections can be mapped to different architecture views. This step involved identifying relevant architecture views such as data, context, hardware, AI model, and functional safety views that can address the identified challenges. The article presents a list of the identified underlying challenges, identified relations between the challenges and a mapping to architecture views. The intention of this work is to highlight once more that requirement specifications and system architecture are interlinked, even for AI-specific specification challenges such as specifying requirements for training data and runtime monitoring.

Guidelines for the development of a critical software under emergency.

Context:During the first wave of the COVID-19 pandemic, an international and heterogeneous team of scientists collaborated on a social project to produce a mechanical ventilator for intensive care units (MVM). MVM has been conceived to be produced and used also in poor countries: it is open-source, no patents, cheap, and can be produced with materials that are easy to retrieve.Objective:The objective of this work is to extract from the experience of the MVM development and software certification a set of lessons learned and then guidelines that can help developers to produce safety–critical devices in similar emergency situations.Method:We conducted a case study. We had full access to source code, comments on code, change requests, test reports, every deliverable (60 in total) produced for the software certification (safety concepts, requirements specifications, architecture and design, testing activities, etc.), notes, whiteboard sketches, emails, etc. We validated both lessons learned and guidelines with experts.Findings:We contribute a set of validated lessons learned and a set of validated guidelines, together with a discussion of benefits and risks of each guideline.Conclusion:In this work we share our experience in certifying software for healthcare devices produced under emergency, i.e. with strict and pressing time constraints and with the difficulty of establishing a heterogeneous development team made of volunteers. We believe that the guidelines will help engineers during the development of critical software under emergency.

Powerful authentication regime applicable to naval OFP integrated development (PARANOID): a vision for non-circumventable code signing and traceability for embedded avionics software

Purpose This paper aims to describe an effort to provide for a robust and secure software development paradigm intended to support DevSecOps in a naval aviation enterprise (NAE) software support activity (SSA), with said paradigm supporting strong traceability and provability concerning the SSA’s output product, known as an operational flight program (OFP). Through a secure development environment (SDE), each critical software development function performed on said OFP during its development has a corresponding record represented on a blockchain. Design/methodology/approach An SDE is implemented as a virtual machine or container incorporating software development tools that are modified to support blockchain transactions. Each critical software development function, e.g. editing, compiling, linking, generates a blockchain transaction message with associated information embedded in the output of a said function that, together, can be used to prove integrity and support traceability. An attestation process is used to provide proof that the toolchain containing SDE is not subject to unauthorized modification at the time said critical function is performed. Findings Blockchain methods are shown to be a viable approach for supporting exhaustive traceability and strong provability of development system integrity for mission-critical software produced by an NAE SSA for NAE embedded systems software. Practical implications A blockchain-based authentication approach that could be implemented at the OFP point-of-load would provide for fine-grain authentication of all OFP software components, with each component or module having its own proof-of-integrity (including the integrity of the used development tools) over its entire development history. Originality/value Many SSAs have established control procedures for development such as check-out/check-in. This does not prove the SSA output software is secure. For one thing, a build system does not necessarily enforce procedures in a way that is determinable from the output. Furthermore, the SSA toolchain itself could be attacked. The approach described in this paper enforces security policy and embeds information into the output of every development function that can be cross-referenced to blockchain transaction records for provability and traceability that only trusted tools, free from unauthorized modifications, are used in software development. A key original concept of this approach is that it treats assigned developer time as a transferable digital currency.

Test-Driven Approach for Safety-Critical Software Development

Nowadays software control large majority of systems that humankind use. Systems that software is used widely, such as transportation, military, medicine and avionics must be safe during operation. Fail in these critical systems may cause catastrophic results (i.e. loss of human life, loss or severe damage to environment or equipment etc.). In order to avoid failure on safety critical systems or at least mitigate risks detailed testing is required. Many safety critical systems are developed with sequential phases and tested with test last approach. But test last approach is not sufficient when requirements are unclear or changed. To overcome weaknesses of test last approach we propose test driven approach for safety critical software development, regulated by IEC 61508 standard. A systematic analysis of IEC 61508 software requirements against proposed approach showed that the approach directly supports some objectives and partially supports some of them. Supported objectives are safety requirement identification, simple and testable software design, verification and validation. In general introduced approach suits regulated software development well and this paper outlines these details.

ВНЕДРЕНИЕ НОВОЙ ВЕРСИИ СТАНДАРТА DO-178С В ПРАКТИКУ СЕРТИФИКАЦИИ ПРОГРАММНОГО ОБЕСПЕЧЕНИЯ

The information is given that JSC “Element”, being a developer of electronic systems for aviation purposes and embedded software, pays great attention to their reliability and dependability, including in-flight conditions. It uses the global principle, which provides for certification of software as a process of documenting the life cycle of this software from the moment of issuance of technical specifications to maintenance in operation. The actual standard describing all stages of the development, testing, and implementation of safe software in the aviation industry for over 30 years has been RTCA DO-178 and its analogs (ED-12, KT-178). DO-178 has come a long way of development and improvement. Its various options at different points in time most closely corresponded to the characteristics of the equipment, but, most importantly, to the capabilities of the software development tools. This article presents the main features and differences of the current version of DO-178C from the previous DO-178B. These differences, or, more precisely, the changes, are divided into non-essential and significant. It is noted that DO-178C differs little from DO-178B in text volume, but the main difference is the addition of other documents. These documents in the complex extend the capabilities of DO-178, bringing it closer to the most modern software development technologies. Such technologies are object-oriented programming and model-oriented software development. The specified technologies should be traceable, and their goals should coincide with the goals of the base document. It is noted that aviation application software developers should consider not only the benefits of implementing the DO-178C but also possible delays and additional costs associated with the use of additional tools and technologies. In general, with proper planning, the DO-178C is key to maintaining competitiveness in the development of critical software for aviation. The State Aviation Administration of Ukraine does not provide adequate support to developers and manufacturers of aviation software. We hope that this situation will improve.

Safety critical software ground rules

Safety critical software development field is one of the active research areas in many industries like automotive, medical, railways, nuclear and aerospace are placing increased value on safety and reliability. Safety critical software systems are those systems whose failure could result in the death or a serious injury to the people’s life, security is one of the important topics in the field of safety-critical systems and it must be addressed completely in order to operate safety critical software successfully. In this paper we present a study about the set of standards and different ground rules to be followed in critical software development practices in different industries and the challenges in applying these standards. We also discuss the role of static analysis and software integrity levels in these standards, similarities in these standards and the set of activities followed in the development process of these standards.

Conceptual basis for developing of trainig models in complex system software assembling generator

This paper presents conceptual basis for developing of training models of interactive assembling system for automatic building of application software systems, obtained during practical works over Design and architecture of software and Object-oriented analysis and design courses. The system is intended for practical demonstration of basic stages and operations in development of complex and critical software. This also includes a consideration of solutions for some substantial problems of complex systems software generation, such as: program module compatibility, formalization of computer interaction and choosing of formal model for human machine interface. In addition, several training model implementations are provided.

Safety Critical Software Development: A Modeling Approach

Safety Critical Software Development: A Modeling Approach

Approach to the development of a Unified Framework for Safety Critical Software Development

Software is increasingly being used to provide functionality in safety critical domains. The complexity involved in the development of software for these domains can bring challenges concerned with safety. International standards are published, providing information on practices which must be implemented in order to satisfy the regulations. This paper details an investigation of the relevant standards that companies need to implement in order to satisfy the regulatory requirements. A literature review was conducted which examines the relevant Quality Management System, Risk Management and Software Development standards across the safety critical domains; and examines the challenges faced by software development organizations with standard and multiple standard implementation. The existing approaches to standards consolidation, integration and harmonization were also examined. To investigate the challenges in implementing these standards, interviews were conducted with a medical device software development company having a Quality Management System in place and beginning to implement the relevant Software Development standards. In addition, an interview was conducted with a consultancy company who has experience in the implementation and maintenance of Quality Management Systems in small and medium enterprises. Following the literature review and interviews conducted, a comparison of the standards was conducted to investigate the similarities and differences in the structure of sections and subsections and to estimate the potential for the further consolidation of these standards. Following the results of the comparisons, a detailed mapping of standard requirements was conducted, mapping requirements of generic and domain specific Quality Management System standards. Future work will focus on the consolidation and integration of standards practices which need to be implemented by companies developing safety critical software and development of Unified Framework that incorporates all necessary requirements to comply with regulatory requirements.

From Place/Transition Petri nets to B abstract machines for safety critical systems

Fulfilling the safety requirements is one of the most serious and problematic issues in critical systems design and safety critical software development. In this context, thanks to their analysis power, formal methods have been widely used in the various stages of the design and the implementation of safety critical systems. However, some methods such as the B method, although well adapted to safety issues, are still poorly used in large scale industrial environment. The purpose of this paper is to present a methodology of Place/Transition Petri nets transformation into B abstract machines enabling an interesting combination of the graphical modeling power of Petri nets and the verification tools of the B method. In fact, translating a Petri net to a B abstract machine can have many advantages such as the generation of code, the integration of safety invariants or the aggregation with other formal models. Therefore, the B verification will enlarge the scope of its applicability by having a new modeling alternative and passing through model transformation. An illustrative example of the transformation is presented for a railway study case.

Model-Based Testing of Safety Critical Real-Time Control Logic Software

The paper presents the experience of the authors in model based testing of safety critical real-time control logic software. It describes specifics of the corresponding industrial settings and discusses technical details of usage of UniTESK model based testing technology in these settings. Finally, we discuss possible future directions of safety critical software development processes and a place of model based testing techniques in it.

Test Driven Development: Advancing Knowledge by Conjecture and Confirmation

Test Driven Development (TDD) is a critical agile software development practice that supports innovation in short development cycles. However, TDD is one of the most challenging agile practices to adopt because it requires changes to work practices and skill sets. It is therefore important to gain an understanding of TDD through the experiences of those who have successfully adopted this practice. We collaborated with an agile team to provide this experience report on their adoption of TDD, using observations and interviews within the product development environment. This article highlights a number of practices that underlie successful development with TDD. To provide a theoretical perspective that can help to explain how TDD supports a positive philosophy of software development, we have revised Northover et al.’s conceptual framework, which is based on a four stage model of agile development, to reinterpret Popper’s theory of conjecture and falsification in the context of agile testing strategies. As a result of our findings, we propose an analytical model for TDD in agile software development which provides a theoretical basis for further investigations into the role of TDD and related practices.

On Using B in the Design of Secure Micro-controllers: An Experience Report

The stepwise formal development of safety critical software is now a well established engineering practice, noticeably in railway systems. However, it has not been applied as successfully to hardware development, where formal methods are mainly used for verification and gate level transformations and optimizations. In this paper, we report our recent experience in the stepwise formal development of a real macro-cell, that opens the way to the design of synchronous digital circuits with zero functional bugs. We propose a development flow suited for obtaining proven correct-by-construction circuits that further possess additional robustness properties desirable for secure chips. The reported work is prospective and is meant to show the feasibility of such a technique for high confidence trustful devices.

Enhancing spark's contract checking facilities using symbolic execution

Spark, a subset of Ada for engineering safety and security-critical systems, is one of the best commercially available frameworks for formal-methods-supported development of critical software. Spark is designed for verification and includes a software contract language for specifying functional properties of procedures. Even though Spark and its static analysis components are beneficial and easy to use, its contract language is rarely used for stating properties beyond simple constraints on scalar values due to the burdens the associated tool support imposes on developers. Symbolic execution (SymExe) techniques have made significant strides in automating reasoning about deep semantic properties of source code. However, most work on SymExe has focused on bug-finding and test case generation as opposed to tasks that are more verification-oriented such as contract checking. In previous work we have presented: (a) SymExe techniques for checking software contracts in embedded critical systems, and (b) Bakar Kiasan, a tool that implements these techniques in an integrated development environment for Spark. In this paper, we give a detailed walk-through of Bakar Kiasan as it is applied to an industrial code base for an embedded security device. We illustrate how Bakar Kiasan provides significant increases in automation, usability, and functionality over existing Spark contract checking tools, and we present results from performance evaluations of its application to industrial examples.

Model-Based Design, Development and Validation for UAS Critical Software

This paper focuses on the major phases present in the development of critical software for UAS: design, development, testing and validation with flying experiments. A model-based approach is the backbone of all these development phases. The presented approach enables the researchers or engineers to work over the whole development cycle with the same tools, and produces an improvement over the classical design cycle. The model-based approach is illustrated with the development of guidance, navigation and control algorithms applied to rotary-wing UAVs. The paper discusses several implementation issues, including the integration of a hardware in the loop (HWIL) test environment within the model-based development cycle. Finally, several simulations and experimental results for this application are shown.

Agile Software Development

AbstractThis focus section contains papers related to agile software development. The papers address a range of research areas including the application of agile methods to safety critical software development, the relationship of agile development with user experience design and how to measure flow in lean software development.

High-integrity agile processes for the development of safety critical software

Typically, safety critical software systems are developed using plan-driven development processes. Agile processes have evolved to help reduce costs of software development and seek to minimise documentation overheads. For safety critical systems that must undergo certification, documentation is essential. The question this paper addresses is: can a process based on agile principles be used to deliver a safety critical software product, but also the evidence needed to satisfy assurance objectives? The paper makes three contributions. Firstly, it presents an analysis of agile processes and their applicability in this domain. It reviews positive indicators for their use, outlines challenges associated with their deployment and proposes strategies for addressing these challenges. Secondly, it makes a number of recommendations for adapting an agile process to the domain. Finally, the paper reports on an experiment to demonstrate the plausibility of using of an adapted agile process for building a safety critical software system.

Experience report

High-level tools have become unavoidable in industrial software development processes. Safety-critical embedded programs don't escape this trend. In the context of safety-critical embedded systems, the development processes follow strict guidelines and requirements. The development quality assurance applies as much to the final embedded code, as to the tools themselves. The French company Esterel Technologies decided in 2006 to base its new SCADE SUITE 6TM certifiable code generator on Objective Caml. This paper outlines how it has been challenging in the context of safety critical software development by the rigorous norms DO-178B, IEC 61508, EN 50128 and such.

Development Life-cycle of Critical Software Under FoCaL

Before their installation, critical systems must be assessed by an independent authority, who ensures that software components are really compliant with a set of requirements described in standards. Such standards describe the framework and the rules to be strictly followed along the development process. Moreover high levels of safety highly recommand the use of formal methods. In this paper, we examine how the FoCaL development environment can help to fulfil these requirements and to ease assessment. This tool aims to help all stages of critical software development, at least when formal methods are required (step-by-step specification and implementation, properties expressed by first-order formulae, proofs helped by automatic tool). Upon our experience as either software safety assessor or researchers in software engineering and formal methods, we propose a development life cycle adapted to the FoCaL specificity and compliant with independent assessment requirements, through a complete example. We show how features such as inheritance, late binding, redefinition, parametrisation, encapsulation and declarations/definitions, properties/theorems, whole development checked by an independent proof assistant and partially automatic documentation can be used to improve the global safety and the re-use of software components.

Model-based solution for developing automotive software

Modern cars are increasingly equipped with electronic control units plus embedded functions to increase the comfort and safety for the driver and occupants. The control-intensive software design of automotive applications in particular poses a challenge. Esterel presents a solution for the development of critical real-time software, especially containing important control logic parts. The focus is on typical errors in the software development life cycle and the solution via using the version 6 SCADE development environment.