Abstract
Modified condition/decision coverage (MC/DC) is a structural code coverage metric, originally defined in the standard DO-178B, intended to be an efficient coverage metric for the evaluation of the testing process of software incorporating decisions with complex Boolean expressions. The upcoming standard ISO 26262 for safety-relevant automotive systems prescribes MC/DC for ASIL D as a highly recommended coverage metric. One assumed benefit of MC/DC is that it requires a much smaller number of test cases in comparison to multiple condition coverage (MCC), while sustaining a quite high error-detection probability. Programming languages like C, commonly used for implementing software for the automotive domain, are using short-circuit evaluation. For short-circuit evaluation the number of test cases for MCC is much smaller than in a non-short-circuit environment because many redundant test cases occur. We evaluated the trade-off between the number of test cases for MCC and MC/DC for a case study from the automotive domain and observed a very low overhead (only 5 %) for the number of test cases necessary for MCC compared to MC/DC. This motivated an analysis of programs containing decisions where the number and structure of the referring Boolean expressions vary. Our results show that the overhead for a test suite for MCC is on the average only about 35 % compared to MC/DC and the maximum overhead is approximately 100 % (for decisions with up to 5 conditions). This means that a test set for MCC is in the worst case around twice as big as a test set for MC/DC for a program with short-circuit evaluation with maximum 5 conditions. Considering the lower error-detection effectiveness of MC/DC compared to MCC, we conclude with the strong recommendation to use MCC as a coverage metric for testing safety-relevant software (with a limited number of conditions) implemented in programming languages with short-circuit evaluation.
Highlights
Safety-relevant software, causing crucial damage to people or the environment when malfunctioning, has to be tested exhaustively to guarantee a high reliability
Regarding this enhanced confidence in the testing process using the multiple condition coverage (MCC)-criterion instead of the Modified condition/decision coverage (MC/Decision coverage (DC))-criterion, and considering the acceptable overhead for the increased number of test cases to achieve full MCC, we question the use of the MC/DC-criterion for safety-relevant software implemented in a programming language with short-circuit evaluation
As we showed in the analysis the number of test cases required for MCC causes only a small overhead (5 % for our case study) for testing in comparison to MC/DC
Summary
Safety-relevant software, causing crucial damage to people or the environment when malfunctioning, has to be tested exhaustively to guarantee a high reliability. The Boolean operators AND and OR of Ada are using eager evaluation (aka: greedy evaluation), i.e. the whole Boolean expression is evaluated to determine the resulting outcome Complete testing of such an expression is given by the metric MCC (multiple condition coverage). This coverage requires all possible Boolean assignments to the input variables of the decision. For a decision containing N Boolean conditions (N = 3 for the example above) we would need to generate 2N inputs to test all possible combinations This means the testing effort grows exponentially with an increasing complexity of the decision.
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
