Construction Of Test Cases Research Articles

Investigating Documented Privacy Changes in Android OS

Android has empowered third-party apps to access data and services on mobile devices since its genesis.This involves a wide spectrum of user privacy-sensitive data, such as the device ID and location. In recent years, Android has taken proactive measures to adapt its access control policies for such data, in response to the increasingly strict privacy protection regulations around the world. When each new Android version is released, its privacy changes induced by the version evolution are transparently disclosed, and we refer to them as documented privacy changes (DPCs). Implementing DPCs in Android OS is a non-trivial task, due to not only the dispersed nature of those access control points within the OS, but also the challenges posed by backward compatibility. As a result, whether the actual access control enforcement in the OS implementations aligns with the disclosed DPCs becomes a critical concern. In this work, we conduct the first systematic study on the consistency between the operational behaviors of the OS at runtime and the officially disclosed DPCs. We propose DopCheck, an automatic DPC-driven testing framework equipped with a large language model (LLM) pipeline. It features a serial of analysis to extract the ontology from the privacy change documents written in natural language, and then harnesses the few-shot capability of LLMs to construct test cases for the detection of DPC-compliance issues in OS implementations. We apply DopCheck with the latest versions (10 to 13) of Android Open Source Project (AOSP). Our evaluation involving 79 privacy-sensitive APIs demonstrates that DopCheck can effectively recognize DPCs from Android documentation and generate rigorous test cases. Our study reveals that the status quo of the DPC-compliance issues is concerning, evidenced by 19 bugs identified by DopCheck. Notably, 12 of them are discovered in Android 13 and 6 in Android 10 for the first time, posing more than 35% Android users to the risk of privacy leakage. Our findings should raise an alert to Android users and app developers on the DPC compliance issues when using or developing an app, and would also underscore the necessity for Google to comprehensively validate the actual implementation against its privacy documentation prior to the OS release.

Exploratory and correlation analysis on the impact of accessibility testing on PC software development projects

The work consists of a theoretical introduction that conceptualizes software engineering, software development, testing, escaped defects, user experience, usability and accessibility. In order to conduct a research in the area of tests focused on accessibility, an analysis of real projects was proposed. Interviews were conducted with managers of software projects that are already in the market, extraction of data from the same projects, exploratory analysis of the data, correlation tests and structuring of the results found. At the end of the research, it was observed the relevance of some data and the significance of the relationship between them. From the results obtained, it is concluded the relevance of the construction of test cases for accessibility, as well as the factors that contribute to the teams to give due attention to accessibility tests, such as team composition and external collaborations.

Comparative Analysis of Constraint Handling Techniques for Constrained Combinatorial Testing

Constraints depict the dependency relationships between parameters in a software system under test. Because almost all systems are constrained in some way, techniques that adequately cater for constraints have become a crucial factor for adoption, deployment and exploitation of Combinatorial Testing (CT). Currently, despite a variety of different constraint handling techniques available, the relationship between these techniques and the generation algorithms that use them remains unknown, yielding an important gap and pressing concern in the literature of constrained combination testing. In this article, we present a comparative empirical study to investigate the impact of four common constraint handling techniques on the efficiency of six representative (greedy and search-based) test suite generation algorithms. The results reveal that the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Verify</i> technique implemented with the Minimal Forbidden Tuple (MFT) approach is the fastest, while the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Replace</i> technique is promising for producing the smallest constrained covering arrays, especially for algorithms that construct test cases one-at-a-time. The results also show that there is an interplay between efficiency of the constraint handler and the test suite generation algorithm into which it is developed.

GNFCVulFinder: NDEF Vulnerability Discovering for NFC-Enabled Smart Mobile Devices Based on Fuzzing

Near-field communication (NFC) is a set of communication protocols that enable two electronic devices. Its security and reliability are welcomed by mobile terminal manufactures, banks, telecom operators, and third-party payment platforms. Simultaneously, it has also drawn more and more attention from hackers and attackers, and NFC-enabled devices are facing increasing threats. To improve the security of the NFC technology, the paper studied the technology of discovering security vulnerabilities of NFC Data Exchange Format (NDEF), the most important data transmission protocol. In the paper, we proposed an algorithm, GTCT (General Test Case Construction and Test), based on fuzzing to construct test cases and test the NDEF protocol. GTCT adopts four strategies to construct test cases, manual, generation, mutation, and “reverse analysis,” which can detect logic vulnerabilities that fuzzing cannot find and improve the detection rate. Based on GTCT, we designed an NDEF vulnerability discovering framework and developed a tool named “GNFCVulFinder” (General NFC Vulnerability Finder). By testing 33 NFC system services and applications on Android and Windows Phones, we found eight vulnerabilities, including DoS vulnerabilities of NFC service, logic vulnerabilities about opening Bluetooth/Wi-Fi/torch, design flaws about the black screen, and DoS of NFC applications. Finally, we give some security suggestions for the developer to enhance the security of NFC.

Software Test Data Generation based on Path Testing using Genetic Algorithms

Test data generation is the task of constructing test cases for predicting the acceptability of novel or updated software. Test data could be the original test suite taken from previous run or imitation data generated afresh specifically for this purpose. The simplest way of generating test data is done randomly but such test cases may not be competent enough in detecting all defects and bugs. In contrast, test cases can also be generated automatically and this has a number of advantages over the conventional manual method. Genetic Algorithms, one of the automation techniques, are iterative algorithms and apply basic operations repeatedly in greed for optimal solutions or in this case, test data. By finding out the most error-prone path using such test cases one can reduce the software development cost and improve the testing efficiency. During the evolution process such algorithms pass on the better traits to the next generations and when applied to generations of software test data they produce test cases that are closer to optimal solutions. Most of the automated test data generators developed so far work well only for continuous functions. In this study, we have used Genetic Algorithms to develop a tool and named it TG-GA (Test Data Generation using Genetic Algorithms) that searches for test data in a discontinuous space. The goal of the work is to analyze the effectiveness of Genetic Algorithms in automated test data generation and to compare its performance over random sampling particularly for discontinuous spaces.

Model-based, Mutation-driven Test-case Generation Via Heuristic-guided Branching Search

This work introduces a heuristic-guided branching search algorithm for model-based, mutation-driven test-case generation. The algorithm is designed towards the efficient and computationally tractable exploration of discrete, non-deterministic models with huge state spaces. Asynchronous parallel processing is a key feature of the algorithm. The algorithm is inspired by the successful path planning algorithm Rapidly exploring Random Trees (RRT). We adapt RRT in several aspects towards test-case generation. Most notably, we introduce parametrized heuristics for start and successor state selection, as well as a mechanism to construct test cases from the data produced during the search. We implemented our algorithm in the existing test-case generation framework MoMuT. We present an extensive evaluation of the proposed heuristics and parameters of the algorithm, based on a diverse set of demanding models obtained in an industrial context. In total, we continuously utilized 128 CPU cores on three servers for several weeks to gather the experimental data presented. We show that branching search works well and the use of multiple heuristics is justified. With our new algorithm, we are now able to process models consisting of over 2,300 concurrent objects. To our knowledge, there is no other mutation-driven test-case generation tool that is able to process models of this magnitude.

A Neural Network Based Strategy (NNBS) for Automated Construction of Test Cases for Testing an Embedded System using Combinatorial Techniques

Testing an embedded system is required to locate bugs in software, diminish risk, development, repairs costs and to improve performance for both users and the company. Embedded software testing tools are useful for catching defects during unit, integration and system testing. Embedded systems in many cases must be optimized by engaging crucial areas of the embedded systems considering all factors of the input domain. The most important concern is to build a place of test cases depend on design of the requirements that can recognize more number of faults at a least rate and point in time in the major sections of an embedded system. This paper proposes a Neural Network Based strategy (NNBS) to generate optimized test cases based on the considerations of the system. A tool called NNTCG (Neural Network Test Case Generator) has been build up based on the method proposed in this paper. Test cases are generated for testing an embedded system using NNTCG and the same are used to determine the expected output through the neural network and the output generated from the actual firmware. The faulty paths within the firmware are determined when the output generated by the neural network is not same as the output generated by the firmware.

Динамический анализ приложений с графическим пользовательским интерфейсом на основе символьного исполнения

In this paper, we explore the possibilities of applying dynamic symbolic execution (or concolic testing) methods to applications with graphical user interfaces. Such applications inherently feature interactive user input processing and multithreaded execution. These features typically decrease the effectiveness of dynamic symbolic execution by increasing the volume of processed code not related to actual application functionality. We present a hybrid approach that combines commonly used GUI test automation methods based on GUI model excavation with dynamic symbolic execution methods to construct test cases for checking internal application logic. We have implemented this approach using two open-source tools - test automation framework GUITAR and Java byte-code static instrumentation framework Coffee Machine. GUI model extracted automatically by GUITAR tool is extended with symbolic traces relevant to application GUI event handlers. Our test generation module for GUITAR combines these symbolic traces into complex queries to be processed by SMT solver. The resulting test cases are valid within automatically extracted GUI structure model and allow to check different execution paths in GUI event handler code. We have checked our hybrid approach on a set of small open-source applications and identified several bugs caused by uncaught exceptions. The paper is concluded with an overview of current limitations and possible improvements of the hybrid approach.

Testing models and model transformations using classifying terms

This paper proposes the use of equivalence partitioning techniques for testing models and model transformations. In particular, we introduce the concept of classifying terms, which are general OCL terms on a class model enriched with OCL constraints. Classifying terms permit defining equivalence classes, in particular for partitioning the source and target model spaces of the transformation, defining for each class a set of equivalent models with regard to the transformation. Using these classes, a model validator tool is able to automatically construct object models for each class, which constitute relevant test cases for the transformation. We show how this approach of guiding the construction of test cases in an orderly, systematic and efficient manner can be effectively used in combination with Tracts for testing both directional and bidirectional model transformations and for analyzing their behavior.

Reduction of Model Checking-based Test Generation using Satisfiability

Constructing test cases from counterexamples generated by a model checker is an important method to perform test automation. In fact, one counterexample may cover multiple test goals, which leads to unnecessary calls to the model checker, and redundant test cases in test suite such that affect the testi ng performance. A method to test suite reduction based on satisfiability is proposed. The kripke model is translated in conjunction with test goals (trap properties) into CNFs. And then test goal t o generate counterexample is selected according to the hardness of the corresponding CNF, after that, model checking the selected test goal to generate counterexample. The generated counterexample is translated in conjunction with those uncovered test goals i nto CNFs. If the corresponding CNF is unsatisfiable then the test goal is pick ed out from the set of test goals. Meanwhile, the new generated test case is winnowed by test suite to reduce the redundancy before it is added into the test suite. Experimental results show that t he method proposed in this paper is effective for reducing the model checker calls and the length of the test suite. At the same time, the coverage and error detection capability of the test suite are not decl ined.

Optimal two-phase vaccine allocation to geographically different regions under uncertainty

In this article, we consider a decision process in which vaccination is performed in two phases to contain the outbreak of an infectious disease in a set of geographic regions. In the first phase, a limited number of vaccine doses are allocated to each region; in the second phase, additional doses may be allocated to regions in which the epidemic has not been contained. We develop a simulation model to capture the epidemic dynamics in each region for different vaccination levels. We formulate the vaccine allocation problem as a two-stage stochastic linear program (2-SLP) and use the special problem structure to reduce it to a linear program with a similar size to that of the first stage problem. We also present a Newsvendor model formulation of the problem which provides a closed form solution for the optimal allocation. We construct test cases motivated by vaccine planning for seasonal influenza in the state of North Carolina. Using the 2-SLP formulation, we estimate the value of the stochastic solution and the expected value of perfect information. We also propose and test an easy to implement heuristic for vaccine allocation. We show that our proposed two-phase vaccination policy potentially results in a lower attack rate and a considerable saving in vaccine production and administration cost.

Study on IPv6 Capabilities of Mobile Devices

With the rapid development of Internet, the current Internet Protocol IPv4 cannot meet the new requirements. Compared with IPv4, IPv6 has more advantages. Fruitful results have been borne in aspects of IPv6 protocol study and network construction in China. On the other hand, application and popularization of IPv6 is far from sufficient. In recent years, mobile devices have popularized very rapidly. More and more people use mobile devices, including smartphones and tablets, to surf online everywhere. If such a gigantic amount of mobile devices can access network with IPv6, it will be very important for application of IPv6. In this paper, we first analyze the market of mobile devices to determine the devices to be tested. Then we introduce how to construct test environment and test cases. After that, we illustrate the test procedure. Then we summarize how the mainstream mobile devices support IPv6. At the end of this paper, we look forward the future work.

Generating asynchronous test cases from test purposes

Context Input/output transition system (IOTS) models are commonly used when next input can arrive even before outputs are produced. The interaction between the tester and an implementation under test (IUT) is usually assumed to be synchronous. However, as the IUT can produce outputs at any moment, the tester should be prepared to accept all outputs from the IUT, or else be able to block (refuse) outputs of the implementation. Testing distributed, remote applications under the assumptions that communication is synchronous and actions can be blocked is unrealistic, since synchronous communication for such applications can only be achieved if special protocols are used. In this context, asynchronous tests can be more appropriate, reflecting the underlying test architecture which includes queues. Objective In this paper, we investigate the problem of constructing test cases for given test purposes and specification input/output transition systems, when the communication between the tester and the implementation under test is assumed to be asynchronous, performed via multiple queues. Method When issuing verdicts, asynchronous tests should take into account a distortion caused by the queues in the observed interactions. First, we investigate how the test purpose can be transformed to account for this distortion when there are a single input queue and a single output queue. Then, we consider a more general problem, when there may be multiple queues. Results We propose an algorithm which constructs a sound test case, by transforming the test purpose prior to composing it with the specification without queues. Conclusion The proposed algorithm mitigates the state explosion problem which usually occurs when queues are directly involved in the composition. Experimental results confirm the resulting state space reduction.

Combinatorial Testing: Principles and Methods

PDF HTML阅读 XML下载 导出引用 引用提醒 组合测试:原理与方法 DOI: 作者: 作者单位: 作者简介: 通讯作者: 中图分类号: 基金项目: Supported by the National Natural Science Foundation of China under Grant Nos.60673044, 60633010 (国家自然科学基金) Combinatorial Testing: Principles and Methods Author: Affiliation: Fund Project: 摘要 | 图/表 | 访问统计 | 参考文献 | 相似文献 | 引证文献 | 资源附件 | 文章评论 摘要:组合测试能够在保证错误检出率的前提下采用较少的测试用例测试系统.但是,组合测试用例集的构造问题的复杂度是NP完全的.组合测试方法的有效性和复杂性吸引了组合数学领域和软件工程领域的学者们对其进行深入的研究.总结了近年来在组合测试方面的研究进展,主要内容包括:组合测试准则的研究、组合测试生成问题与其他NP完全问题的联系、组合测试用例的数学构造方法、采用计算机搜索的组合测试生成方法以及基于组合测试的错误定位技术. Abstract:Combinatorial testing can use a small number of test cases to test systems while preserving fault detection ability. However, the complexity of test case generation problem for combinatorial testing is NP-complete. The efficiency and complexity of this testing method have attracted many researchers from the area of combinatorics and software engineering. This paper summarizes the research works on this topic in recent years. They include: various combinatorial test criteria, the relations between the test generation problem and other NP-complete problems, the mathematical methods for constructing test cases, the computer search techniques for test generation and fault localization techniques based on combinatorial testing. 参考文献 相似文献 引证文献

Injecting rapid feedback and collaborative reasoning in teaching specifications

We describe an approach to teaching formal interface specifications using aspects of the Collaborative Reasoning Paradigm . The module requires students to construct test cases independently and cooperatively based on their understanding of a given set of method specifications. Students are supported by software-based reasoning assistants that guide them through their exercises and provide realtime feedback as they work --- both for the students and the instructor. We describe the design of the course module, the supporting reasoning assistant, and representative reasoning exercises. We conclude with a discussion of evaluation results from a recent pilot study conducted at Clemson University.

Comparative Study of Three-Dimensional Wing Drag Minimization by Different Optimization Techniques

The main goal of this paper is to document a comparative study of different computational-fluid-dynamics-based optimization techniques applied to the solution of a three-dimensional wing drag minimization problem. To achieve this objective, three optimization tools were used: SYN107 (Intelligent Aerodynamics International), MDOPT (The Boeing Company), and OPTIMAS (Israel Aerospace Industries). The first tool employs gradient-based search techniques using the continuous adjoint equation, the second one is a response-surface method, and the last one uses a floating-point genetic algorithm as its search engine. As the starting geometry, the public domain DPW-W1 wing (a test case for the Third AIAA Drag Prediction Workshop) was used. The comparisons included herein are provided in three stages: cross analysis of the initial geometry by the computational fluid dynamics tools employed in the optimizations, optimization of the initial geometry to minimum drag, and cross analysis of optimal shapes achieved by the optimization tools using all computational fluid dynamics tools employed. The cross analysis also includes results from an independent computational fluid dynamics method that was not used in any of the optimization efforts. These results help quantify the level of variation that is inherent in, and can be expected from, application of the current state-of-the-art aerodynamic optimization methods. The present work may be regarded as a move toward the construction of reliable test cases for an aerodynamic shape optimization problem. Another goal of this collaborative investigation is to collect lessons learned from this pilot project to help develop a model for an aerodynamic optimization workshop.

On the identification of categories and choices for specification-based test case generation

The category-partition method and the classification-tree method help construct test cases from specifications. In both methods, an early step is to identify a set of categories (or classifications) and choices (or classes). This is often performed in an ad hoc manner due to the absence of systematic techniques. In this paper, we report and discuss three empirical studies to investigate the common mistakes made by software testers in such an ad hoc approach. The empirical studies serve three purposes: (a) to make the knowledge of common mistakes known to other testers so that they can avoid repeating the same mistakes, (b) to facilitate researchers and practitioners develop systematic identification techniques, and (c) to provide a means of measuring the effectiveness of newly developed identification techniques. Based on the results of our studies, we also formulate a checklist to help testers detect such mistakes.

Using static analysis to improve automatic test generation

Conformance testing is still the main industrial validation technique for telecommunication protocols. In practice, the automatic construction of test cases based on finite-state models is hindered by the state explosion problem. We try to reduce its magnitude by using static analysis techniques in order to obtain smaller but equivalent models.

An experience in integrating automated unit testing practices in an introductory programming course

Unit testing is one of the core practices in the Extreme Programming lightweight software development method, and it is usually carried out with the help of software frameworks that ease the construction of test cases as an integral part of programming tasks. This work describes our first results in studying the integration of automated unit testing practices in conventional 'introduction to programming' laboratories. Since the work used a classical procedural language in the course's assignments, we had to design a specific testing framework called tpUnit. The results of the experiment points out that a straightforward approach for the integration of unit testing in first-semester courses do not result in the expected outcomes in terms of student's engagement in the practice.

Classification-tree restructuring methodologies: a new perspective

The classification-tree method developed by Grochtmann et al. (1993, 1995) provided a useful approach for constructing test cases from functional specifications. It was automated by Chen and Poon (1997, 1998) through their tree construction methodology. In a follow-up study, Chen and Poon found that the effectiveness of constructing legitimate test cases could be improved under certain circumstances via a classification-tree restructuring algorithm. We develop another tree restructuring algorithm to cater for other situations not covered previously. The algorithms complement each other. We also compare the relative effectiveness of these algorithms and provide guidelines on applying them in practice.