Individual Test Cases Research Articles

Spectrum-based rule- and item-level localization of faults in context-free grammars

We describe and evaluate spectrum-based methods aimed at finding faults in context-free grammars. In their basic form, they take as input a test suite and a parser for the grammar that is modified to collect grammar spectra (i.e., the sets of grammar elements used in attempts to parse the individual test cases), and return as output a ranked list of suspicious elements. We define grammar spectra suitable for localizing faults on the level of the grammar rules (i.e., rule spectra) and the rules’ individual symbols (i.e., item spectra), respectively. We show how both types of grammar spectra can be collected by both LL and LR parsers, and how the JavaCC, ANTLR, and CUP parser generators can be modified and used to automate the collection of the grammar spectra. We also show how grammar spectra can be synthesized directly from test cases derived from a grammar, and how such synthetic spectra can be used to localize differences between a grammar and a black-box system under test.We first evaluate our approach over a large number of medium-sized single fault grammars, which we constructed by fault seeding from a common origin grammar. At the rule level, it ranks the rules containing the seeded faults within the top five rules in about 40%–70% of the cases, depending on the applied parsing technique, test suite, and ranking metric, and pinpoints them (i.e., correctly identifies them as unique most suspicious rule) in about 10%–30% of the cases, with significantly better results for the synthetic spectra. At the item level, our approach remains remarkably effective despite the larger number of possible locations, provided it is coupled with a simple tie-breaking strategy that prefers items with the right-most designated position over other items from the same rules in a tie. It typically ranks the seeded faults within the top five positions in about 30%–60% of the cases, and pinpoints them in about 15%–40% of the cases. This specialized item-level localization also significantly outperforms a simplistic extension of the rule-level localization, where all positions within a rule are given the same score.We further evaluate our approach over grammars that contain real faults. We show that an iterative method can be used to localize and manually remove one by one multiple faults in grammars submitted by students enrolled in various compiler engineering courses; in most iterations, the top-ranked rule already contains an error, and no error is ranked outside the top five ranked rules. We finally apply our approach to a large open-source SQLite grammar and show where this version deviates from the language accepted by the actual SQLite system.

HETerogeneous vectorized or Parallel (HETPv1.0): an updated inorganic heterogeneous chemistry solver for the metastable-state NH4+–Na+–Ca2+–K+–Mg2+–SO42−–NO3−–Cl−–H2O system based on ISORROPIA II

Abstract. We describe a new Fortran computer program to solve the system of equations for the NH4+–Na+–Ca2+–K+–Mg2+–SO42-–NO3-–Cl−–H2O system, based on the algorithms of ISORROPIA II. Specifically, the code solves the system of equations describing the “forward” (gas + aerosol input) metastable state but with algorithm improvements and corrections. These algorithm changes allow the code to deliver more accurate solution results in formal evaluations of accuracy of the roots of the systems of equations, while reducing processing time in practical applications by about 50 %. The improved solution performance results from several implementation improvements relative to the original ISORROPIA algorithms. These improvements include (i) the use of the “interpolate, truncate and project” (ITP) root-finding approach rather than bisection, (ii) the allowance of search interval endpoints as valid roots at the onset of a search, (iii) the use of a more accurate method to solve polynomial subsystems of equations, (iv) the elimination of negative concentrations during iterative solutions, (v) corrections for mass conservation enforcement, and (vi) several code structure improvements. The new code may be run in either a “vectorization” mode wherein a global convergence criterion is used across multiple tests within the same chemical subspace or a “by case-by-case” mode wherein individual test cases are solved with the same convergence criteria. The latter approach was found to be more efficient on the compiler tested here, but users of the code are recommended to test both options on their own systems. The new code has been constructed to explicitly conserve the input mass for all species considered in the solver and is provided as open-source Fortran shareware.

Quantification of Regression Test Suite Execution Time in Parallel Execution Setup with Weighted Test Suite Split Algorithm

Regression test suite execution time study focus is essentially on two aspects. They are execution time reduction and making effective use of available hardware resources and manpower. This paper investigates how the regression test suite can be split into subsets to make use of parallel execution across several machines with identical execution speeds and asymmetrical execution speeds. In the symmetrical execution speed setup, long test execution time test cases are evenly distributed across all the hardware machines. However, in asymmetrical execution speed machines, more test cases are distributed to speed machines to make efficient use of hardware resources. In all the situations where there is automation tool to execute the individual test cases of test suite this approach can be employed to make effective use of hardware resources and to keep the execution time within bounds. The Algorithms can also be used in situations where there are queues involved and serial, fixed time service takes place for each of the entity being served.

Quantification of Regression Test Suite Execution Time in Parallel Execution Setup with Weighted Test Suite Split Algorithm

Regression test suite execution time study focus is essentially on two aspects. They are execution time reduction and making effective use of available hardware resources and manpower. This paper investigates how the regression test suite can be split into subsets to make use of parallel execution across several machines with identical execution speeds and asymmetrical execution speeds. In the symmetrical execution speed setup, long test execution time test cases are evenly distributed across all the hardware machines. However, in asymmetrical execution speed machines, more test cases are distributed to speed machines to make efficient use of hardware resources. In all the situations where there is automation tool to execute the individual test cases of test suite this approach can be employed to make effective use of hardware resources and to keep the execution time within bounds. The Algorithms can also be used in situations where there are queues involved and serial, fixed time service takes place for each of the entity being served.

Understanding medical professionalism using express team-based learning; a qualitative case-based study

ABSTRACT Medical Professionalism (MP) defined as values, behaviours and attitudes that promote professional relationships, public trust and patient safety is a vital competency in health profession education. MP has a distinctive uniqueness due to cultural, contextual, conceptual, and generational variations. There is no standard instructional strategy to probe the understanding of MP in a cohesive, structured, interactive manner. This study aimed to investigate undergraduate medical students’ understanding of MP using express team-based learning (e-TBL) at both campuses of Royal College of Surgeons Ireland (RCSI). Using the key principles of a sociocultural theoretical lens in adult learning theory, we designed e-TBL as a context-learning-based educational strategy. We conducted three e-TBL sessions on cross-cultural communication and health disparities, a reflective report on clinical encounters, and professionalism in practice. We collected, collated, and analyzed the student experiences qualitatively using data gathered from team-based case discussions during e-TBL sessions. A dedicated working group developed very short-answer questions for the individual readiness assurance test (IRAT) and MP-based case scenarios for team discussions. In this adapted 4-step e-TBL session, pre-class material was administered, IRAT was undertaken, and team-based discussions were facilitated, followed by facilitator feedback. A qualitative inductive thematic analysis was performed, which generated subthemes and themes illustrated in excerpts. Our thematic analysis of data from 172 students (101 from Bahrain and 71 from Dublin) yielded four unique themes: incoming professional attitudes, transformative experiences, sociological understanding of professionalism, and new professional identity formation. This qualitative study provides a deeper understanding of medical students’ perceptions of medical professionalism. The generated themes resonated with divergent and evolving elements of MP in an era of socioeconomic and cultural diversity, transformative experiences, and professional identity formation. The core elements of these themes can be integrated into the teaching of MP to prepare fit-to-practice future doctors.

Test case simplification based on coupling metrics in software bug location

Software test cases are one of the most critical aspects of software testing in the product development process. As software products are updated several times, the same test requirement may be covered by multiple test cases, so this aspect is often redundant, yet the approximate test case set has an impact on its error detection rate. This study proposes the idea of using redundant test cases in software error location, introduces a coupling metric, analyses its program slicing and establishes a second coverage criterion in order to balance the relationship between the reduced test suite and the false detection rate the test case set. The results show that the size of test set and the number of error detection by the Ruby On Rails (ROR) method used in this study are larger than those of other commonly used reduction algorithms. The test suite has the lowest error detection loss rate, with an average of 17.96 % across the six test case sets. The highest error detection capability of individual test cases was found in the reduced test set, with a mean value of 90.63 % in the test set. The method also has the highest average reduction efficiency of 91.05 %. Compared with other simplification methods, the research method has a better balance between the size and false detection rate of the reduced test suite and the advantages of simplification.

Designing a Reliable UAV Architecture Operating in a Real Environment

The article presents a method of designing a selected unmanned aerial platform flight scenario based on the principles of designing a reliable (Unmanned Aerial Vehicle) UAV architecture operating in an environment in which other platforms operate. The models and results presented relate to the medium-range aerial platform, subject to certification under the principles set out in aviation regulations. These platforms are subject to the certification process requirements, but their restrictions are not as restrictive as in the case of manned platforms. Issues related to modeling scenarios implemented by the platform in flight are discussed. The article describes the importance of Functional Hazard Analysis (FHA) and Fault Trees Analysis (FTA) of elements included in the hardware and software architecture of the system. The models in Unified Modeling Language (UML) used by the authors in the project are described, supporting the design of a reliable architecture of flying platforms. Examples of the transformations from user requirements modeled in the form of Use Cases to platform operation models based on State Machines and then to the final UAV operation algorithms are shown. Principles of designing system test plans and designing individual test cases to verify the system’s operation in emergencies in flight are discussed. Methods of integrating flight simulators with elements of the air platform in the form of Software-in-the-Loop (SIL) models based on selected algorithms for avoiding dangerous situations have been described. The presented results are based on a practical example of an algorithm for detecting an air collision situation of two platforms.

Fight silent horror unit test methods by consulting a TestWizard

AbstractTests, when not correctly implemented, can pass on incorrect system implementations rather than fail. In this case, they are named silent horrors or false‐negative tests. They make releasing low‐quality (buggy) versions of the software system more probable. Furthermore, faithfully implementing test specifications is crucial when they play the role of documentation, like when documenting components or services or driving legacy systems' re‐engineering. This paper presents TestWizard, a novel approach and tool for automatically assessing individual tests' quality from the point of view of their coherence to specifications. TestWizard automatically assesses the quality of each individual test case w.r.t. its specification, providing detailed reports on why a single test is a false negative, hence helping testers fix them. Thus, TestWizard can help to automate the test code review process, which is still mainly manual today. The analysis of 1012 test implementations, developed by 123 students in three experiments, shows that TestWizard is (1) by far more accurate than code review performed by multiple students, (2) slightly better than code review performed by three senior experts, and (3) always able to detect a significant percentage of false‐negative test methods (up to 21.22%).

Live Evaluation of Auditory Preference, a Laboratory Test for Evaluating Auditory Preference.

Many laboratory tests are performed under unrealistic conditions. Tasks, such as repeating words or sentences, are performed in simple loudspeaker setups. Currently, many research groups focus on realistic audiovisual laboratory setups. Fewer groups focus on the tasks performed during testing. A semicontrolled laboratory test method focusing on the tasks performed, the Live Evaluation of Auditory Preference (LEAP) was evaluated. LEAP is developed to evaluate hearing-instrument performance in test scenarios that represent everyday listening situations. LEAP was evaluated in a feasibility study. The method comprises conversations between a test participant and one or two test leaders, enabling evaluation of the test participant's own voice. The method allows for visual cues (when relevant) and introduce social pressure to participate in the conversation. In addition, other everyday listening tasks, such as watching television (TV) and listening to radio, are included. In this study, LEAP was used to assess preference for two hearing aid settings using paired comparisons. Nineteen experienced hearing aid users (13 females and 6 males; mean age 74 years), participated in the study. LEAP was performed at three visits to the laboratory. In addition, participants conducted a field trial where the two hearing aid programs were compared using Ecological Momentary Assessments (EMA). During LEAP testing, six mandatory test cases were used, representing commonly occurring everyday listening situations. Individual test cases were also included, selected from individually experienced listening situations during the field trial. Within- and between-session reliability of the LEAP test was investigated. Validity was investigated by comparing the LEAP and the EMA results. For the current signal-processing evaluation, the test was judged to have acceptable reliability and validity. The inclusion of individually selected test cases increased the representativeness of the LEAP test, but it did not substantially alter the results in the current study. LEAP in its current implementation seems suitable for investigating signal-processing preference in the laboratory in a way that is indicative of everyday preference. The LEAP method represents one step forward in bringing the real world into the laboratory.

Using metamorphic relations to verify and enhance Artcode classification

Software testing is often hindered where it is impossible or impractical to determine the correctness of the behaviour or output of the software under test (SUT), a situation known as the oracle problem. An example of an area facing the oracle problem is automatic image classification, using machine learning to classify an input image as one of a set of predefined classes. An approach to software testing that alleviates the oracle problem is metamorphic testing (MT). While traditional software testing examines the correctness of individual test cases, MT instead examines the relations amongst multiple executions of test cases and their outputs. These relations are called metamorphic relations (MRs): if an MR is found to be violated, then a fault must exist in the SUT. This paper examines the problem of classifying images containing visually hidden markers called Artcodes, and applies MT to verify and enhance the trained classifiers. This paper further examines two MRs, Separation and Occlusion, and reports on their capability in verifying the image classification using one-way analysis of variance (ANOVA) in conjunction with three other statistical analysis methods: t-test (for unequal variances), Kruskal–Wallis test, and Dunnett’s test. In addition to our previously-studied classifier, that used Random Forests, we introduce a new classifier that uses a support vector machine, and present its MR-augmented version. Experimental evaluations across a number of performance metrics show that the augmented classifiers can achieve better performance than non-augmented classifiers. This paper also analyses how the enhanced performance is obtained.

Automated Test Data Generation Based on a Genetic Algorithm with Maximum Code Coverage and Population Diversity

In the present paper, we investigate an approach to intelligent support of the software white-box testing process based on an evolutionary paradigm. As a part of this approach, we solve the urgent problem of automated generation of the optimal set of test data that provides maximum statement coverage of the code when it is used in the testing process. We propose the formulation of a fitness function containing two terms, and, accordingly, two versions for implementing genetic algorithms (GA). The first term of the fitness function is responsible for the complexity of the code statements executed on the path generated by the current individual test case (current set of statements). The second term formulates the maximum possible difference between the current set of statements and the set of statements covered by the remaining test cases in the population. Using only the first term does not make it possible to obtain 100 percent statement coverage by generated test cases in one population, and therefore implies repeated launch of the GA with changed weights of the code statements which requires recompiling the code under the test. By using both terms of the proposed fitness function, we obtain maximum statement coverage and population diversity in one launch of the GA. Optimal relation between the two terms of fitness function was obtained for two very different programs under testing.

Boundary layer vortex sheet evolution around an accelerating and rotating cylinder

Abstract

Nondeterministic Analysis of Monopropellant Catalyst Bed Model

Nondeterministic analyses were performed on a one-dimensional, steady, chemically reacting, reduced-order model for hydrogen peroxide flow through a monopropellant catalyst bed. Polynomial chaos expansion was implemented to decrease sampling time in order to perform the probabilistic analyses including quantification of global sensitivities using Sobol indices, construction of axial property uncertainty envelopes for random inputs, and construction of posterior probability distributions with credible intervals for variation in chemical tuning parameters. Results from the study show that model behavior is primarily governed by propellant mass fraction and activation energy of the global reaction. Additionally, posterior distributions generated via Bayesian inference indicate that activation energy and the number of active sites per volume are related by a logarithmic family of solutions as a result of the reaction advancement gradient form in the model. A region of high probability activation energy and number of active sites per volume values are identified using Markov chain Monte Carlo to compare model predictions with experimental data across five individual test cases.

A Fuzzy Rule Based Approach for Test Case Selection Probability Estimation in Regression Testing

Regression testing is a very essential activity during maintenance of software. Due to constraints of time and cost, it is not possible to re-execute every test case with respect to every change occurred. Thus, a technique is required that selects and prioritises the test cases efficiently. This paper proposes a novel fuzzy rule-based approach for selecting and ordering a number of test cases from an existing test suite to predict the selection probability of test cases using multiple factors. The test cases, which have ability to find high fault detection rate with maximum coverage and minimum execution time to test are selected. The results specify the effectiveness of the proposed model for predicting the selection probability of individual test cases.

A fuzzy rule-based approach for test case selection probability estimation in regression testing

Regression testing is a very essential activity during maintenance of software. Due to constraints of time and cost, it is not possible to re-execute every test case with respect to every change occurred. Thus, a technique is required that selects and prioritises the test cases efficiently. This paper proposes a novel fuzzy rule-based approach for selecting and ordering a number of test cases from an existing test suite to predict the selection probability of test cases using multiple factors. The test cases, which have ability to find high fault detection rate with maximum coverage and minimum execution time to test are selected. The results specify the effectiveness of the proposed model for predicting the selection probability of individual test cases.

A community-driven reconstruction of the Aspergillus niger metabolic network

BackgroundAspergillus niger is an important fungus used in industrial applications for enzyme and acid production. To enable rational metabolic engineering of the species, available information can be collected and integrated in a genome-scale model to devise strategies for improving its performance as a host organism.ResultsIn this paper, we update an existing model of A. niger metabolism to include the information collected from 876 publications, thereby expanding the coverage of the model by 940 reactions, 777 metabolites and 454 genes. In the presented consensus genome-scale model of A. niger iJB1325 , we integrated experimental data from publications and patents, as well as our own experiments, into a consistent network. This information has been included in a standardized way, allowing for automated testing and continuous improvements in the future. This repository of experimental data allowed the definition of 471 individual test cases, of which the model complies with 373 of them. We further re-analyzed existing transcriptomics and quantitative physiology data to gain new insights on metabolism. Additionally, the model contains 3482 checks on the model structure, thereby representing the best validated genome-scale model on A. niger developed until now. Strain-specific model versions for strains ATCC 1015 and CBS 513.88 have been created containing all data used for model building, thereby allowing users to adopt the models and check the updated version against the experimental data. The resulting model is compliant with the SBML standard and therefore enables users to easily simulate it using their preferred software solution.ConclusionExperimental data on most organisms are scattered across hundreds of publications and several repositories.To allow for a systems level understanding of metabolism, the data must be integrated in a consistent knowledge network. The A. niger iJB1325 model presented here integrates the available data into a highly curated genome-scale model to facilitate the simulation of flux distributions, as well as the interpretation of other genome-scale data by providing the metabolic context.

Void fraction measurements of bubbly flow in a horizontal 90° bend using wire mesh sensors

Wire mesh sensors were used to investigate the void fraction distribution of two-phase gas-liquid bubbly flow in a 50.8 mm diameter horizontal test section containing a 90° bend. The 90° bend had a radius of curvature of 76.2 mm, and its inlet was installed at an axial distance of one hundred pipe diameters from the test section's inlet. Wire mesh sensors measurements were recorded at axial locations of 97.5, 112.8, and 130.8 pipe diameters from the air-water inlet, which included axial locations both upstream and downstream of the 90° bend. Deionised water and compressed air were used as the working fluids, with the bubbly flow regime achieved at superficial liquid velocities ranging from 3.50 m/s to 5.42 m/s and superficial gas velocities, referenced at atmospheric conditions, ranging from 0.10 to 1.25 m/s. The effects of the 90° bend, as well as superficial gas and liquid velocities, on the void fraction distribution were parametrically investigated. Extensive measurements of the area-averaged, time-averaged, and bulk-averaged void fraction are reported for a total of 27 individual test cases. Radial profiles of the time-averaged void fraction are presented for all test cases along both vertical and horizontal axes. Results demonstrated the effects of superficial liquid and gas velocities on phase distribution before and after the bend. Statistical techniques were applied to discretize a transitional bubbly-slug flow pattern into constituent small bubble and large bubble flow patterns.

An Efficient DAGbM-KSJS Algorithm for Agile Software Testing

Agile software testing is a software testing practice that follows the principles of agile software development. In this paper, an optimal Agile software testing is performed using Directed Acyclic Graph-based Model (DAGbM). Initially, the suggested model pre-processes the test case dataset, then by deploying the Dependency Assessment for Use Case (DAUC) algorithm, the dependency between the uses of cases are determined. The K-Shingling based Jaccard Similarity (KSJS) algorithm estimates the similarity among every test cases and prioritizes the clustered test cases using Span Clustering based Prioritization (SCP) algorithm. After prioritizing the clustered test cases, the use cases are prioritized based on dependency. Finally, the minimum distance value is exploited for prioritizing the individual test cases. The performance of the suggested method is validated using parameters such as code average, failure rate, prioritization time, and percentage of defects detected. The validation results prove that when compared to the existing methods, the suggested method provides optimal results for all the parameters.

Open Source Code Coverage Tools for Java: A Comparative Analysis

This research compares two code coverage tools to understand the relationship between the code coverage and regression testing, henceforth the effectiveness of the code coverage detail provided by the tools. The methodology adopted to meet the objectives follows theoretical as well as empirical approach. To achieve the empirical approach a platform was setup in eclipse IDE for Java application which was integrated with Junit to execute test cases for Java program. Two open source code coverage tools CodeCover and Eclemma were exercised respectively upon a small Java application with twenty one test cases. This execution shows that EclEmma is effective in providing the detail of individual test case. CodeCover on the other side provides combined measurement of the test cases. CodeCover provide the coverage at fine level as well as at coarser level of granularity. However regression testing demands detail coverage made by each test case which code cover fails to dispense. EclEmma generates code coverage report by providing information about individual test case. This information is most desirable when performing test case optimization in regression testing. Further coverage details of test suites given by code coverage tool will be used in proposing a hybrid regression test case optimization technique.

A metamorphic testing approach for supporting program repair without the need for a test oracle

Test suite based automated program repair (APR) relies on a test oracle to determine the execution result of individual test cases. The applicability of APR techniques, therefore, is limited by the fact that test oracles may not exist. Metamorphic Testing (MT) is a testing approach that, rather than checking the correctness of individual test outputs, checks testing results through verification of relations among multiple test cases and their outputs: MT can therefore be applied without test oracles. This paper presents an integration of MT with APR that enables application of APR without the need for a test oracle. Two important issues for this integration which have been thoroughly investigated and addressed are: (1) feasibility — which is addressed by proposing a framework to support the integration, and then presenting MT-GenProg, a tool incorporating MT with the popular APR technique GenProg; and (2) effectiveness — which is confirmed through an empirical study of GenProg and MT-GenProg on 1,143 program versions from the IntroClass benchmark suite, demonstrating MT-GenProg’s comparable performance to GenProg, in terms of repair effectiveness. We conclude that the proposed integration is both practically feasible and effective, and thus successfully extends APR techniques to a broader application domain.