Testing Aspect-oriented Programs Research Articles

RAMBUTANS: automatic AOP-specific test generation tool

Aspect-oriented programming (AOP) is a programmatic methodology to handle better modularized code by separating crosscutting concerns from the traditional abstraction boundaries. Automated testing, as one of the most demanding needs of the software development to reduce both human effort and costs, is a delicate issue in testing aspect-oriented programs. Prior studies in the automated test generation for aspect-oriented programs have been very limited with respect to the need for both adequate tool support and capability concerning effectiveness and efficiency. This paper describes a new AOP-specific tool for testing aspect-oriented programs, called RAMBUTANS. The RAMBUTANS tool uses a directed random testing technique that is especially well suited for generating tests for aspectual features in AspectJ. The directed random aspect of the tool is parameterized by associating weights to aspects, advice, methods, and classes by controlling object and joint point creations during the test generation process. We present a comprehensive empirical evaluation of our tool against the current AOP test generation approaches on three industrial aspect-oriented projects. The results of the experimental and statistical tests showed that RAMBUTANS tool produces test suites that have higher fault-detection capability and efficiency for AspectJ-like programs.

Automated test generation technique for aspectual features in AspectJ

ContextAspect-oriented programming (AOP) has been promoted as a means for handling the modularization of software systems by raising the abstraction level and reducing the scattering and tangling of crosscutting concerns. Studies from literature have shown the usefulness and application of AOP across various fields of research and domains. Despite this, research shows that AOP is currently used in a cautious way due to its natural impact on testability and maintainability. ObjectiveTo realize the benefits of AOP and to increase its adoption, aspects developed using AOP should be subjected to automated testing. Automated testing, as one of the most pressing needs of the software industry to reduce both effort and costs in assuring correctness, is a delicate issue in testing aspect-oriented programs that still requires advancement and has a way to go before maturity. MethodPrevious attempts and studies in automated test generation process for aspect-oriented programs have been very limited. This paper proposes a rigorous automated test generation technique, called RAMBUTANS, with its tool support based on guided random testing for the AspectJ programs. ResultsThe paper reports the results of a thorough empirical study of 9 AspectJ benchmark programs, including non-trivial and larger software, by means of mutation analysis to compare RAMBUTANS and the four existing automated AOP testing approaches for testing aspects in terms of fault detection effectiveness and test effort efficiency. The results of the experiment and statistical tests supplemented by effect size measures presented evidence of the effectiveness and efficiency of the proposed technique at 99% confidence level (i.e. p<0.01). ConclusionThe study showed that the resulting randomized tests were reasonably good for AOP testing, thus the proposed technique could be worth using as an effective and efficient AOP-specific automated test generation technique.

Towards Semantic Mutation Testing of Aspect-Oriented Programs

Aspect-oriented programs have received much attention from software testing researchers. Various testing techniques and approaches have been proposed to tackle issues and challenges when testing aspect-oriented programs including traditional mutation testing. In traditional mutation testing of aspect-oriented programs, mutants are generated by making small changes to the syntax of the aspect-oriented language. Recently, a new approach known as semantic mutation testing has been proposed. This approach mutates the semantics of the language in which the program is written. The mutants generated misunderstandings of the language which are different classes of faults. Aspect-oriented programming presents itself with different properties that can be further explored with respect to semantic mutation testing. This paper describes various possible scenarios that semantic mutation testing strategy might have particular value in testing aspect-oriented programs.

Status and Awards

This issue has three interesting papers about test design. The first paper in this issue, Automated Functional Testing of Online Search Services, by Zhou, Zhang, Hagenbuchner, Tse, Kuo, and Chen, tests search engines. A major problem that is addressed is the ‘oracle function’, which for these applications means how can we know whether the search results are ‘correct’ (recommended by Byoungju Choi). The second, Automated Verification and Testing of User-interactive Undo Features in Database Applications, by Ngo and Tan, addresses the problem of testing the ability for users to undo operations. A correspondence between program statements that raise erroneous effects and program statements that can undo those effects is reported and is used to develop a verification technique (recommended by Shaoying Liu). The third, Testing Aspect-oriented Programs with Finite State Machines, by Xu, El-Ariss, Xu, and Wang, reports that aspect-oriented programs have new kinds of faults, which they call aspect faults. This observation is used to develop a test strategy to detect these kinds of faults (recommended by Sudipto Ghosh). The journal held its third editorial board meeting at the Fifth International Conference on Software Testing, Verification, and Reliability (ICST) in Montreal. It was a good meeting with our publishing editor from Wiley and about a dozen members of the board. This meeting is important to resolve issues, discuss strategic directions for the journal, and plan for future directions. Our most important decision was to institute two yearly awards for the journal. The first will be a Best Paper of the Year award. Accepted papers will be nominated by the Reviewing Editor who handles the paper, and then the winner will be chosen from among the nominees by votes of members of the editorial board. The second will be the paper with the Highest 5-year Impact. We will make a short list based on the number of references and then a discussion and vote from a subset of the editorial board. We expect to announce the first winners early in the next year.

Empirical evaluation of the fault detection effectiveness and test effort efficiency of the automated AOP testing approaches

Context Testing process is a time-consuming, expensive, and labor-intensive activity in any software setting including aspect-oriented programming (AOP). To reduce the testing costs, human effort, and to achieve the improvements in both quality and productivity of AOP, it is desirable to automate testing of aspect-oriented programs as much as possible. Objective In recent past years, a lot of research effort has been devoted to testing aspect-oriented programs but less effort has been dedicated to the automated AOP testing. This denotes that the current research on automated AOP testing is not sufficient and is still in a stage of infancy. In order to advance the state of the research in this area and to provide testers of AOP-based projects with a comparison basis, a detailed evaluation of the current automated AOP testing approaches in a thorough and experimental manner is required. Thus, the objective of this paper is to provide such evaluation of the current approaches. Method In this paper, we carry out an empirical study based on mutation analysis to examine four (namely Wrasp, Aspectra, Raspect, and EAT) existing automated AOP testing approaches, particularly their underlying test input generation and selection strategies, with regard to fault detection effectiveness. In addition, the approaches are compared in terms of required effort in detecting faults as part of efficiency evaluation. Results The experimental results and comparison provided insights into the effectiveness and efficiency of automated AOP testing with their respective strengths and weaknesses. Results showed that EAT is more effective than the other automated AOP testing approaches but not significant for all approaches. EAT was found to be significantly better than Wrasp at 95% confidence level (i.e. p < 0.05), but not significantly better than Aspectra or Raspect. Concerning the test effort efficiency, Wrasp was significantly ( p < 0.05) efficient with requiring the lowest amount of test effort compared to the other approaches. Whereas, EAT showed to be not very efficient by recording the highest amount of test effort. Conclusion This implies that EAT can currently be the most effective automated AOP testing approach but perhaps less efficient. More generally, search-based testing (as underlying strategy of EAT approach) might achieve better effectiveness but at the cost of greater test effort compared to random testing (as underlying strategy of other approaches).

Automated State-Based Unit Testing for Aspect-Oriented Programs: A Supporting Framework.

Interactions between aspects and classes are a new source for faults. Existing objectoriented testing techniques are not adequate for testing aspect-oriented programs. As a consequence, new testing techniques must be developed. We present, in this paper, a state-based unit testing technique for aspect-oriented programs and associated tool (AJUnit). The technique focuses on the integration of one or several aspects to a class. The objective is to ensure that the integration is done without affecting the original behavior of the class. AJUnit, based on the model of JUnit, generates testing sequences covering an aspect(s)-class block of code. It also supports the execution and verification of the generated sequences. We focus on AspectJ programs. Testing an aspect(s)-class block is done incrementally. Furthermore, the generated sequences are archived. In the case of a change instantiated on a class or on one of its related aspects, only the testing sequences corresponding to the affected parts of the code are retested. The same approach is followed when introducing a new aspect influencing the class under test. The technique is based on several testing criteria that we defined. The generation and verification process of the testing sequences is completely automated.