Required Test Coverage Research Articles

X-Masking for Deterministic In-System Tests

Deterministic in-system tests begin to play an essential role in safety-critical applications, in large data centers, or in monitoring silicon aging, to name just a few. All of these ecosystems require periodic, high-quality tests to assure required test coverage and short test application, especially in designs that must test themselves during system operations. In order for deterministic tests to be in-system applicable, they should compact multi-million-bit test responses with unknown (X) values to small signatures. This, in turn, allows for a much faster input-only streaming and a simultaneous reduction of the on-chip-stored test data volume, a system memory, and test time. Typically, the unknown states, whose sources vary from uninitialized memories to unpredictable last-minute timing violations, render signatures unusable. Hence, test response compaction requires some form of protection. This paper presents a user-tunable X-masking scheme. It works synergistically with on-chip test compression logic by employing encoded test data to completely filter out unknown values that otherwise might reach a test response compactor such as a MISR or test result sticky-bits used by the on-chip compare framework. It makes the proposed scheme a very versatile of its kind. Experimental results obtained for several industrial cores show feasibility and efficiency of the proposed scheme altogether with actual impact of X-masking on various test-related statistics.

Fundamental Principles, Processes, and Roles of Environmental Qualification Test Strategy for Complex Engineered Systems

“In real life, strategy is actually very straightforward. You pick a general direction and implement it like hell.” Jack Welch Former CEO of General Electric. A qualification strategy, created and executed on complex systems, enables effective and efficient environmental testing at the component, unit, section and system levels. Cost, schedule, and technical challenges highlight the need for an environmental qualification test strategy that provides a robust test regime to verify the design meets user requirements, balanced with a timesaving approach. This paper explains three fundamentals to follow when creating an environmental qualification test strategy that cover risk reduction testing, test planning, and test design. Based on lessons learned through two decades of results, the strategies described in the fundamentals provide an optimized approach to qualification by; 1) demonstrating test environment capabilities as risk reduction, 2) defining key stakeholders, roles and responsibilities along with methods to reduce the possibility of increased test durations due to setup inefficiencies, and 3) providing methods to ensure the required test coverage and test methodology is robust enough to detect design deficiencies. This strategy aligns with customer desire to reduce test delays while at the same time not sacrificing test thoroughness.

Prioritized Process Test: An Alternative to Current Process Testing Strategies

Testing processes and workflows in information and Internet of Things systems is a major part of the typical software testing effort. Consistent and efficient path-based test cases are desired to support these tests. Because certain parts of software system workflows have a higher business priority than others, this fact has to be involved in the generation of test cases. In this paper, we propose a Prioritized Process Test (PPT), which is a model-based test case generation algorithm that represents an alternative to currently established algorithms that use directed graphs and test requirements to model the system under test. The PPT accepts a directed multigraph as a model to express priorities, and edge weights are used instead of test requirements. To determine the test-coverage level of test cases, a test-depth-level concept is used. We compared the presented PPT with five alternatives (i.e. the Process Cycle Test (PCT), a naive reduction of test set created by the PCT, Brute Force algorithm, Set-covering-Based Solution and Matching-based Prefix Graph Solution) for edge coverage and edge-pair coverage. To assess the optimality of the path-based test cases produced by these strategies, we used 14 metrics based on the properties of these test cases and 59 models that were created for three real-world systems. For all edge coverage, the PPT produced more optimal test cases than the alternatives in terms of the majority of the metrics. For edge-pair coverage, the PPT strategy yielded similar results to those of the alternatives. Thus, the PPT strategy is an applicable alternative as it reflects both the required test coverage level and the business priority in parallel.

Employment of multiple algorithms for optimal path-based test selection strategy

Executing various sequences of system functions in a system under test represents one of the primary techniques in software testing. The natural way to create effective, consistent and efficient test sequences is to model the system under test and employ an algorithm to generate the tests that satisfy a defined test coverage criterion. Several criteria of test set optimality can be defined. In addition, to optimize the test set from an economic viewpoint, the priorities of the various parts of the system model under test must be defined. Using this prioritization, the test cases exercise the high priority parts of the system under test more intensely than those with low priority. Evidence from the literature and our observations confirm that finding a universal algorithm that produces an optimal test set for all test coverage and test set optimality criteria is a challenging task. Moreover, for different individual problem instances, different algorithms provide optimal results. In this paper, we present a path-based strategy to perform optimal test selection. The strategy first employs a set of current algorithms to generate test sets; then, it assesses the optimality of each test set by the selected criteria, and finally, chooses the optimal test set. The experimental results confirm the validity and usefulness of this strategy. For individual instances of 50 system under test models, different algorithms provided optimal results; these results varied by the required test coverage level, the size of the priority parts of the model, and the selected test set optimality criteria.

Functional decomposition—A contribution to overcome the parameter space explosion during validation of highly automated driving

Objective: Particular testing by functional decomposition of the automated driving function can potentially contribute to reducing the effort of validating highly automated driving functions. In this study, the required size of test suites for scenario-based testing and the potential to reduce it by functional decomposition are quantified for the first time.Methods: The required size of test suites for scenario-based approval of a so-called Autobahn-Chauffeur (SAE Level 3) is analyzed for an exemplary set of scenarios. Based on studies of data from failure analyses in other domains, the possible range for the required test coverage is narrowed down and suitable discretization steps, as well as ranges for the influence parameters, are assumed. Based on those assumptions, the size of the test suites for testing the complete system is quantified. The effects that lead to a reduction in the parameter space for particular testing of the decomposed driving function are analyzed and the potential to reduce the validation effort is estimated by comparing the resulting test suite sizes for both methods.Results: The combination of all effects leads to a reduction in the test suites’ size by a factor between 20 and 130, depending on the required test coverage. This means that the size of the required test suite can be reduced by 95–99% by particular testing compared to scenario-based testing of the complete system.Conclusions: The reduction potential is a valuable contribution to overcome the parameter space explosion during the validation of highly automated driving. However, this study is based on assumptions and only a small set of exemplary scenarios. Thus, the findings have to be validated in further studies.

TEST SCENARIOS GENERATION USING PATH COVERAGE

Testing is one of the very important component of software development process. Properly generated test sequences may not only locate the defects in software, but also help in reducing the high cost associated with software testing. It is often desired that test sequences should be automatically generated to achieve required test coverage. Automatic test sequence generation is a major problem in software testing. The aim of this study is to generate test sequences for source code using ModelJunit. ModelJUnit is a extended library of JUnit. We have generate automatic test sequences and some testing criterion coverage such as node coverage, edge coverage and edge pair coverage. This paper describes a systematic test sequence generation technique using the path based approach.

Optimisation of software testing using Genetic Algorithm

Software testing is meant to increase confidence in the correctness of software. Test data generation is one of the key issues in software testing. A properly generated test suite may not only locate the errors in a software system, but also help in reducing the high cost associated with software testing. It is often desired that test data in the form of test sequences within a test suite can be automatically generated to achieve the required test coverage. This paper proposes Genetic Algorithm (GA) to test data generation for optimising the software testing.