Small Number Of Test Cases Research Articles

Improving the evaluation of change requests using past cases

As one of the leading causes of project failures, requirements changes are inevitable in any software project. Hence, we propose an intelligent approach to facilitate the risk analysis of a change request by providing information about past cases found in similar change requests, the solutions adopted, and a support tool. The proposed approach uses case-based reasoning to retrieve previous cases similar to the current case. This approach also uses association rules to analyze patterns in the dataset and calculate the probability of risks associated with change requests. We prepared a case study to validate the proposal by analyzing the most frequent challenges in change management and considering how it can solve or minimize such problems. Results show that the proposed approach successfully assists decision-making, predicts potential risks, and suggests coherent solutions to the user. We have developed a support tool to evaluate this approach in practice with experts and obtained four different outcomes. Only a small set of cases failed to provide relevant results to the user. The use of case-based reasoning and association rules has proven to be advantageous in change management despite validity threats associated with the small number of test cases and experts involved.

On the Application of Quick Artificial Bee Colony Algorithm (qABC) for Attenuation of Test Suite in Real-Time Software Applications

Software testing plays a vital role during the software development process, as it ensures quality software deployment. Success of software testing depends on the design of effective test cases. To achieve the optimization of generated test cases, the proposed approach combines both global and local searches by means of intelligent agents which exhibit the behaviour of employed bees, onlooker bees, and scout bees in the qABC algorithm. The proposed qABC algorithm has key improvements over the basic artificial bee colony algorithm (ABC) in test optimization by reducing redundancy, filtering of test cases in each iteration and parallel working of the bees. Further, the fitness evaluation of the test cases is done by employing two test adequacy metrics namely path coverage and mutation score. Further, the experimental evaluation of qABC, GA, and the basic ABC based test cases is done using several case study applications. The result shows that qABC outperforms the other algorithms in terms of effectiveness of test cases in revealing the faults with less time and a smaller number of test cases.

Automatic Generation and Optimization of Combinatorial Test Cases from UML Activity Diagram Using Particle Swarm Optimization

Generation of test cases is one of the essential activities of the software testing process. The process of executing a programme to identify defects to improve the system's quality is known as software testing. Manually writing test cases takes time, effort, and money. On the other hand, generating test cases automatically is the solution to this problem. For this automation process, a model-based test case generation technique would be acceptable. A model is usually required to generate test cases in the model-based testing technique. Nowadays, researchers have relied on the activity diagram to generate test cases. Test cases for combinatorial logic systems are required. Combinatorial testing is essential for producing a small number of test cases and identifying errors occurred by interactions between system input parameters. Information about constraints, parameters and its values are required for generation of test cases. It is difficult to extract information regarding constraints, parameters, its values, and interactions between parameters from an Activity Diagram. A novel approach is proposed to extract this information from an Activity Diagram. The authors created a tool that automatically generates combinatorial test cases using UML Activity Diagrams. The proposed tool has two main parts. First, the combinatorial test design model is developed for extraction of input parameters. Second part is generation of optimized number of combinatorial test cases using Particle Swarm Optimization algorithm. Finally, the authors experimented on a real-world case study namely viz. Railway Reservation using the proposed tool, and it is shown that the proposed tool generated optimum number of combinatorial test cases.

Optimization Method of Web Fuzzy Test Cases Based on Genetic Algorithm

In order to solve the problems of low code coverage, few vulnerabilities found, and poor fuzzing effect caused by the small number of test cases and single types in Web fuzzing, on the basis of studying the current Web fuzzing methods, the existing fuzzing Web applications are tested Program research. A genetic algorithm-based method for optimizing fuzzing test cases for Web applications is proposed. It analyzes and counts the traffic of public network website business with Web service attack characteristics, and uses genetic algorithms to generate a large number of test cases with various types to explore the Web service vulnerability that exists. Based on the creation of a Web attack signature database with weights, this method uses genetic algorithms to randomly pre-generate the test cases of the fuzzing test, and uses the response of the Web service to repeatedly iterate the weights of different attack signatures in the Web attack signature database. So as to generate the best test cases. Experimental analysis shows that this method effectively finds security vulnerabilities in Web applications.

Optimizing the Software Testing Problem Using Search-Based Software Engineering Techniques

Software testing is a fundamental step in the software development lifecycle. Its purpose is to evaluate the quality of software applications. Regression testing is an important testing methodology in software testing. The purpose of regression testing is to validate the software after each change of its code. This involves adding new test cases to the test suite and running the test suite as the software changes, making the test suite larger. The cost and time of the project are affected by the test suite size. The challenge is to run regression testing with a smaller number of test cases and larger amount of software coverage. Minimization of the test suite with maximization of the software coverage is an NP-complete problem. Search-based software engineering is an important topic in software engineering, which addresses software engineering optimization problems to find the optimal/approximate solution of the given problem. This study investigated an approach to reducing the regression testing effort and saving time. It also solved the regression testing optimization problem by achieving the maximum test suite coverage with the minimum test suite size. Several experiments were conducted to obtain the optimal solutions for the regression testing problem. We propose an optimization methodology that combines a genetic algorithm and a greedy algorithm to optimize regression testing by respectively maximizing the software test coverage and minimizing the test suite size. The proposed methodology can conveniently deliver fault-free, fully covered, and powerful programs for mission-critical functions. It can be applied to test a real-time system that has high requirements for reliability, security, and safety.

Investigation into the effectiveness of white‐box T‐way testing

An unduly large number of test cases are required for effective testing of programmes containing complex decision statements. In this context, modified condition/decision coverage (MC/DC) testing has been acknowledged to provide effective testing using a test suite whose size is linear in the number of the clauses present in a predicate. MC/DC testing is well-accepted and is mandated by several testing standards. T-way testing is another prominent testing technique that helps to limit the combinatorial explosion of black-box test cases. Its application to white-box testing promises to provide effective testing with a comparatively small number of test cases. The authors empirically investigate the effectiveness of MC/DC testing vis-a-vis white-box pairwise, 3-way and 4-way testing.

Leveraging Textual Specifications for Grammar-Based Fuzzing of Network Protocols

Grammar-based fuzzing is a technique used to find software vulnerabilities by injecting well-formed inputs generated following rules that encode application semantics. Most grammar-based fuzzers for network protocols rely on human experts to manually specify these rules. In this work we study automated learning of protocol rules from textual specifications (i.e. RFCs). We evaluate the automatically extracted protocol rules by applying them to a state-of-the-art fuzzer for transport protocols and show that it leads to a smaller number of test cases while finding the same attacks as the system that uses manually specified rules.

Assessment of the Lagrange Discrete Ordinates Equations for Three-Dimensional Neutron Transport

The Lagrange Discrete Ordinates (LDO) equations, developed by Ahrens as an alternative to the traditional discrete ordinates formulation, have been implemented in Denovo, a three-dimensional radiation transport code developed by Oak Ridge National Laboratory. The LDO equations retain the formal structure of the classical discrete ordinates equations but treat particle scattering in a different way. Solutions of the LDO equations have an interpolatory structure such that the angular flux can be naturally evaluated at directions other than the discrete ordinates used in arriving at the solutions, and the ordinates themselves may be chosen in a strategic way for the problem under consideration. Of particular interest is that the LDO equations have been shown to mitigate ray effects at increased angular resolutions. In this paper we present scalar flux solutions of the LDO equations for a small number of test cases of interest and compare the results against flux solutions generated using standard quadrature types. The LDO equations’ flux solutions were found to be comparable to those resultant from the standard quadrature types in value; results from the LDO equations were also found to be commensurate with those of standard quadrature types when comparing the flux solutions in the context of the experimental benchmark test case examined.

Assessment of algorithms for computing moist available potential energy

Atmospheric moist available potential energy (MAPE) has traditionally been defined as the potential energy of a moist atmosphere relative to that of the adiabatically sorted reference state defining a global potential energy minimum. Although the Munkres algorithm can in principle find such a reference state exactly, its computational cost has prompted much interest in developing heuristic methods for computing MAPE in practice. Comparisons of the accuracy of such approximate algorithms have so far been limited to a small number of test cases; this work provides an assessment of the performance of the algorithms across a wide range of atmospheric soundings, in two different locations. We determine that the divide‐and‐conquer algorithm is the best suited to practical application, but suffers from the previously unexplored shortcoming that it can produce a reference state with higher potential energy than the actual state, resulting in a negative value of MAPE. Additionally, we show that it is possible to construct an algorithm exploiting a previously derived theoretical expression linking MAPE to Convective Available Potential Energy (CAPE). This approach has a similar accuracy to existing approximate sorting algorithms, whilst providing greater insight into the physical source of MAPE. In light of these results, we discuss possible ways to improve on the construction of Available Potential Energy (APE) theory for a moist atmosphere.

Mutomvo: Mutation testing framework for simulated cloud and HPC environments

Many current applications provide high performance to process large volumes of data. These applications usually run in highly distributed environments, like cloud and HPC systems. Nevertheless, the large and complex architectures required for deploying these applications may not be available during the development phase. This limitation can be overcome by using simulation platforms to model a wide range of distributed system configurations and execute these applications in the modeled system. Usually, these applications are tested against a small number of test cases that are manually designed by the testers. It is desirable to have effective test suites in order to detect failures in the application models. In this paper we propose a mutation testing framework for detecting errors in distributed applications executed in simulated environments. The execution of a test suite against the set of mutated models allows to determine its effectiveness for detecting different errors. The proposal has been implemented in a tool called MuTomVo. In order to support the feasibility of the proposal, we have carried out a case study over three applications running in different distributed systems: a client/server model, intensive computation and scientific pipeline.

Testing Domain Dependent Software Reliability Growth Models

Software Reliability Growth Models (SRGMs) are supporting software industries in expecting and scrutinizing quality of software. Numerous SRGMs have been proposed; majority of which concentrate on testing period of software. For testing, domain specific knowledge plays a very crucial role. Based on necessity condition, a set of programmes are in testing phase of software development. “Domain testing is a software technique in which small number of test cases is selected for trial. These sets of testing paths, all of which are to be eventually influenced by designed test cases are called the testing domain which expands with the progress of testing”. Keeping this concept in mind, we propose SRGMs with the concept of testing domain with exponential coverage. Utility of proposed framework has been emphasized in this paper through some models pertaining to different distribution i.e Exponential, Logistic, Weibull and Rayleigh. Moreover, the data analysis is performed to find the estimates of parameters by fitting the models on authentic data sets.

Diversity maximization speedup for localizing faults in single-fault and multi-fault programs

Fault localization is useful for reducing debugging effort. Such techniques require test cases with oracles, which can determine whether a program behaves correctly for every test input. Although most fault localization techniques can localize faults relatively accurately even with a small number of test cases, choosing the right test cases and creating oracles for them are not easy. Test oracle creation is expensive because it can take much manual labeling effort (i.e., effort needed to decide whether the test cases pass or fail). Given a number of test cases to be executed, it is challenging to minimize the number of test cases requiring manual labeling and in the meantime achieve good fault localization accuracy. To address this challenge, this paper presents a novel test case selection strategy based on Diversity Maximization Speedup (Dms). Dms orders a set of unlabeled test cases in a way that maximizes the effectiveness of a fault localization technique. Developers are only expected to label a much smaller number of test cases along this ordering to achieve good fault localization results. We evaluate the performance of Dms on 2 different types of programs, single-fault and multi-fault programs. Our experiments with 411 faults from the Software-artifact Infrastructure Repository show (1) that Dms can help existing fault localization techniques to achieve comparable accuracy with on average 67 and 6 % fewer labeled test cases than previously best test case prioritization techniques for single-fault and multi-fault programs, and (2) that given a labeling budget (i.e., a fixed number of labeled test cases), Dms can help existing fault localization techniques reduce their debugging cost (in terms of the amount of code needed to be inspected to locate faults). We conduct hypothesis test and show that the saving of the debugging cost we achieve for the real C programs are statistically significant.

Reasonability of MC/DC for safety-relevant software implemented in programming languages with short-circuit evaluation

Modified condition/decision coverage (MC/DC) is a structural code coverage metric, originally defined in the standard DO-178B, intended to be an efficient coverage metric for the evaluation of the testing process of software incorporating decisions with complex Boolean expressions. The upcoming standard ISO 26262 for safety-relevant automotive systems prescribes MC/DC for ASIL D as a highly recommended coverage metric. One assumed benefit of MC/DC is that it requires a much smaller number of test cases in comparison to multiple condition coverage (MCC), while sustaining a quite high error-detection probability. Programming languages like C, commonly used for implementing software for the automotive domain, are using short-circuit evaluation. For short-circuit evaluation the number of test cases for MCC is much smaller than in a non-short-circuit environment because many redundant test cases occur. We evaluated the trade-off between the number of test cases for MCC and MC/DC for a case study from the automotive domain and observed a very low overhead (only 5 %) for the number of test cases necessary for MCC compared to MC/DC. This motivated an analysis of programs containing decisions where the number and structure of the referring Boolean expressions vary. Our results show that the overhead for a test suite for MCC is on the average only about 35 % compared to MC/DC and the maximum overhead is approximately 100 % (for decisions with up to 5 conditions). This means that a test set for MCC is in the worst case around twice as big as a test set for MC/DC for a program with short-circuit evaluation with maximum 5 conditions. Considering the lower error-detection effectiveness of MC/DC compared to MCC, we conclude with the strong recommendation to use MCC as a coverage metric for testing safety-relevant software (with a limited number of conditions) implemented in programming languages with short-circuit evaluation.

Evaluating and Comparing Fault-Based Testing Strategies for General Boolean Specifications: A Series of Experiments

A great amount of fault-based testing strategies have been proposed to generate test cases for detecting certain types of faults in Boolean specifications. However, most of the previous studies on these strategies were focused on the Boolean expressions in the disjunctive normal form (DNF), even the irredundant DNF (IDNF)-little work has been conducted to comprehensively investigate their performance on general Boolean specifications. In this study, we conducted a series of experiments to evaluate and compare 18 fault-based testing strategies using over 4000 randomly generated fault-seeded Boolean expressions. In the experiments, a testing strategy is regarded as effective and efficient if it can detect most of the seeded faults using a small number of test cases. Our experimental results show that if a testing strategy is highly effective and efficient when testing the Boolean expressions in the IDNF, it also shows high effectiveness and efficiency on general Boolean expressions. It is found that one family of fault-based testing strategies, namely MUMCUT, normally deliver the best performance among all the 18 strategies. Our study provides an in-depth understanding and insight of fault-based testing for general Boolean expressions.

Application of Orthogonal Experimental Design for the Automatic Software Testing

According to inputting different combination of conditions so as to produce different impacts, software testing designs a large number of test cases. If the implementation of an overall test, due to the limit of the combination of conditions, it is difficult to carry out. In order to generate high quality test cases as early as possible to improve the efficiency of software testing, it is designed a generation tool of the automatic software testing case on orthogonal experimental design. For the test data, the use of that tool design test cases. The practice shows that a small number of test cases are generated, the error detection ability is strong, and it greatly improves the efficiency of software testing.

Cognitive radio engine parametric optimization utilizing Taguchi analysis

Cognitive radio (CR) engines often contain multiple system parameters that require careful tuning to obtain favorable overall performance. This aspect is a crucial element in the design cycle yet is often addressed with ad hoc methods. Efficient methodologies are required in order to make the best use of limited manpower, resources, and time. Statistical methods for approaching parameter tuning exist that provide formalized processes to avoid inefficient ad hoc methods. These methods also apply toward overall system performance testing. This article explores the use of the Taguchi method and orthogonal testing arrays as a tool for identifying favorable genetic algorithm (GA) parameter settings utilized within a hybrid case base reasoning/genetic algorithm CR engine realized in simulation. This method utilizes a small number of test cases compared to traditional design of experiments that rely on full factorial combinations of system parameters. Background on the Taguchi method, its drawbacks and limitations, past efforts in GA parameter tuning, and the use of GA within CR are overviewed. Multiple CR metrics are aggregated into a single figure-of-merit for quantification of performance. Desirability functions are utilized as a tool for identifying ideal settings from multiple responses. Kiviat graphs visualize overall CR performance. The Taguchi method analysis yields a predicted best combination of GA parameters from nine test cases. A confirmation experiment utilizing the predicted best settings is compared against the predicted mean, and desirability. Results show that the predicted performance falls within 1.5% of the confirmation experiment based on 9 test cases as opposed to the 81 test cases required for a full factorial design of experiments analysis.

A strategic approach to social impact measurement of social enterprises

PurposeThe purpose of this paper is to develop a coherent and robust methodology for social impact measurement of social enterprises (SEs) that would provide the conceptual and practical bases for training and embedding.Design/methodology/approachThe paper presents a holistic impact measurement model for SEs, called social impact for local economies (SIMPLEs). The SIMPLE impact model and methodology have been tried and tested on over 40 SEs through a series of three day training courses, and a smaller number of test cases for embedding. The impact model offers a five‐step approach to impact measurement called SCOPE IT; MAP IT; TRACK IT; TELL IT and EMBED IT. These steps help SE managers to conceptualise the impact problem; identify and prioritise impacts for measurement; develop appropriate impact measures; report impacts and embed the results in management decision making.FindingsPreliminary qualitative feedback from participants reveals that while the SIMPLE impact training delivers positive learning experiences on impact measurement and prompts, in the majority of cases, the intensions to implement impact systems, the majority feels the need for follow up embedding support. Paricipant's see value in adopting the SIMPLE approach to support business planning processes. Feedback from two SEs which has receives in‐house facilitates embedding support clearly demonstrates the benefits of working closely with an organisation's staff team to enable effective implementation.Research limitations/implicationsSome key future research challenges are identified as follows: systematically research progress in implementation after training for those participants that do not have facilitated embedding; to further test and develop embedding processes and models (using SIMPLE and other methods) with more SE organisations to identify best practices.Originality/valueThe SIMPLE fills a gap as a tool for holistic impact thinking that offers try and test accessible steps, with robust measures. The innovative steps take SEs through all key impact thought processes from conceptualisation to embedding guidance, feeding into business planning and strategic decision‐making processes. The comparison between the limitations of stand alone impact training and the benefits of facilitated embedding processes is instructive.

Combinatorial Testing: Principles and Methods

PDF HTML阅读 XML下载 导出引用 引用提醒 组合测试:原理与方法 DOI: 作者: 作者单位: 作者简介: 通讯作者: 中图分类号: 基金项目: Supported by the National Natural Science Foundation of China under Grant Nos.60673044, 60633010 (国家自然科学基金) Combinatorial Testing: Principles and Methods Author: Affiliation: Fund Project: 摘要 | 图/表 | 访问统计 | 参考文献 | 相似文献 | 引证文献 | 资源附件 | 文章评论 摘要:组合测试能够在保证错误检出率的前提下采用较少的测试用例测试系统.但是,组合测试用例集的构造问题的复杂度是NP完全的.组合测试方法的有效性和复杂性吸引了组合数学领域和软件工程领域的学者们对其进行深入的研究.总结了近年来在组合测试方面的研究进展,主要内容包括:组合测试准则的研究、组合测试生成问题与其他NP完全问题的联系、组合测试用例的数学构造方法、采用计算机搜索的组合测试生成方法以及基于组合测试的错误定位技术. Abstract:Combinatorial testing can use a small number of test cases to test systems while preserving fault detection ability. However, the complexity of test case generation problem for combinatorial testing is NP-complete. The efficiency and complexity of this testing method have attracted many researchers from the area of combinatorics and software engineering. This paper summarizes the research works on this topic in recent years. They include: various combinatorial test criteria, the relations between the test generation problem and other NP-complete problems, the mathematical methods for constructing test cases, the computer search techniques for test generation and fault localization techniques based on combinatorial testing. 参考文献 相似文献 引证文献

Toward A Theory of Test Case Reduction in Specification Based Software Testing

During the software development, numbers of mistakes are committed by software developers consequential to the insertion of a number of faults in the program. The behavior of a faulty program may be different from expected one. Since testing to detect all imaginable faults is impossible because of large numbers of test cases are required. Fault based testing strategies detects only pre-defined types of faults. Kuhn's fault class hierarchies provide the focus of fault-based testing strategies on detecting particular faults. We have purposed supplementary fault detection strategies by considering Coupling Effect Hypothesis and the Competent Programmer Hypothesis to detect faults with small number of test cases. Our results extend Kuhn's fault class hierarchy combine with Black's fault detection strategies that provide a focus on testing strategies for detection of faults. The resulting tests are also effective for detecting faults in other classes with the sane test cases.

A scalable method for testing real-time systems

Real-time systems (RTSs) are used in different domains such as telephone switching systems, air traffic control systems and patient monitoring systems. The behavior of RTSs is time-sensitive; that is, RTSs interact with their environment with input and output events under time constraints. The violation of such time constraints is the main cause of the misbehavior of RTSs, and may result in severe damage to human lives and the environment [Mandrioli, D., Morasca, S., & Morzenti, A. 1995. ACM Transactions on Computer Systems, 13(4), 365---398]. To prevent failures in RTSs, we must verify that the implementation of an RTS is correct before its deployment. Testing is one of the formal techniques that can be used to achieve this goal. It consists of three main phases: test generation, test execution, and test results analysis. This paper presents a test case generation method for RTSs modeled as Timed Input Output Automata (TIOA). The approach is made in two steps. First, the TIOA describing the system being tested is sampled to construct a subautomaton, which is easily testable (i.e., easy to generate test cases from it). Then, the resulting subautomaton is traversed to generate test cases. Our method is scalable in the sense that it generates a small number of test cases even when the specifications are significant. Moreover, the test cases derived by our method are executable (i.e., they can be run on any error-free implementation of the system being tested).