Model-based Testing Research Articles

A Three-Stage Cellular Automata Model of Complex Large Roundabout Traffic Flow, with a Flow-Efficiency- and Safety-Enhancing Strategy.

Intelligent transportation systems (ITSs) present new opportunities for enhanced traffic management by leveraging advanced driving behavior sensors and real-time information exchange via vehicle-based and cloud-vehicle communication technologies. Specifically, onboard sensors can effectively detect whether human-driven vehicles are adhering to traffic management directives. However, the formulation and validation of effective strategies for vehicle implementation rely on accurate driving behavior models and reliable model-based testing; in this paper, we focus on large roundabouts as the research scenario. To address this, we proposed the Three-Stage Cellular Automata (TSCA) model based on empirical observations, dividing the vehicle journey over roundabouts into three stages: entrance, following, and exit. Furthermore, four optimization strategies were developed based on empirical observations and simulation results, using the traffic efficiency, delay time, and dangerous interaction frequency as key evaluation indicators. Numerical tests reveal that dangerous interactions and delays primarily occurred when the roundabout Road Occupancy Rate (ρ) ranged from 0.12 to 0.24, during which times the vehicle speed also decreased rapidly. Among the strategies, the Path Selection Based on Road Occupancy Rate Recognition Strategy (Simulation 4) demonstrated the best overall performance, increasing the traffic efficiency by 15.65% while reducing the delay time, dangerous interactions, and frequency by 6.50%, 28.32%, and 38.03%, respectively. Additionally, the Entrance Facility Optimization Strategy (Simulation 1) reduced the delay time by 6.90%. While space-based optimization strategies had a more moderate overall impact, they significantly improved the local traffic efficiency at the roundabout by approximately 25.04%. Our findings hold significant practical value, particularly with the support of onboard sensors, which can effectively detect non-compliance and provide real-time warnings to guide drivers in adhering to the prescribed traffic management strategies.

The comparative role of physical system processes in Hudson Strait ice stream cycling: a comprehensive model-based test of Heinrich event hypotheses

Abstract. Despite their recognized significance on global climate and extensive research efforts, the mechanism(s) driving Heinrich events remain(s) a subject of debate. Here, we use the 3D thermomechanically coupled glacial systems model (GSM) to examine the Hudson Strait ice stream surge cycling and the role of three factors previously hypothesized to play a critical role in Heinrich events: ice shelves, glacial isostatic adjustment, and sub-surface ocean temperature forcings. In contrast to all previous modeling studies examining HEs, the GSM uses a transient last glacial cycle climate forcing, global viscoelastic glacial isostatic adjustment model, and sub-glacial hydrology model. The results presented here are based on a high-variance sub-ensemble retrieved from North American history matching for the last glacial cycle. Over our comparatively wide sampling of the potential parameter space (52 ensemble parameters for climate forcing and process uncertainties), we find two modes of Hudson Strait ice streaming: classic binge–purge versus near-continuous ice streaming with occasional shutdowns and subsequent surge onset overshoot. Our model results indicate that large ice shelves covering the Labrador Sea during the last glacial cycle only occur when extreme calving restrictions are applied. The otherwise minor ice shelves provide insignificant buttressing for the Hudson Strait ice stream. While sub-surface ocean temperature forcing leads to minor differences regarding surge characteristics, glacial isostatic adjustment does have a significant impact. Given input uncertainties, the strongest controls on ice stream surge cycling are the poorly constrained deep geothermal heat flux under Hudson Bay and Hudson Strait and the basal drag law. Decreasing the geothermal heat flux within available constraints and/or using a Coulomb sliding law instead of a Weertman-type power law leads to a shift from the near-continuous streaming mode to the binge–purge mode.

Автоматизований метод перевірки та налагодження тестових сценаріїв на основі формальної моделі

Introduction. Model-based test cases generation is a popular strategy for test automation. It helps to reduce time spent on the development of a test suite and can improve level of coverage. However, many reports show shortage of such test cases in poor quality and doubtable efficiency. Purpose. The main goal of the proposed method is cost-effective validation, assessment, debugging and concretization of generated test cases. The method helps improve quality and efficiency of the test cases, make their scenario meaningful and goal-oriented. The method also develops debugging facilities and simplifies data dependency analysis and test scenario editing. Methods. We propose an automated post-processing method which allows to evaluate path that is examined by the test case, and to make safe changes to the path which will eliminate the shortcomings while leaving the coverage targets of the test case unharmed. The method is based on visualization of the path along the control flow graph of the model with additional information about factual evaluation history of all variables and possible alternative variants of behavior. For consistent substitution of certain values in the signal parameters, which would determine the artifacts of the test environment (such as, for example, files, databases, etc.) and check boundary cases (in predicates of conditions, indexing of arrays, etc.), a method of interactive specification of symbolic traces has been developed. Results. The role of the user in deciding whether to add a test case to the project test suite and make changes to it remains crucial, but to reduce labor intensity, the following processes are automated: evaluation of test scenarios according to certain objective characteristics (level of coverage, ability to detect defects, data cohesion, etc.); highlighting of possible alternatives for making corrections; consistent updating of computations for the corresponding corrections. A prototype was developed based on the proposed methods. The empirical results demonstrated a positive impact on the overall efficiency (ability to detect defects and reduce resource consumption) and quality (meaningfulness, readability, maintenance, usefulness for debugging, etc) of the generated test suites. The method allows to make automatically generated test cases trustable and usable. Conclusion. The proposed toolkit significantly reduces the time spent on researching the results of test generation and validation of the obtained tests and their editing. Unlike existing simulation methods, the proposed method not only informs about the values of variables, but also explores the history of their computations and additionally provides information about admissible alternatives. Further we plan to improve the process of localizing the causes of test failure at execution phase to speed up the search for defects.

Model independent refusal trace testing

Software Testing is normally one of the main forms of verification and validation used in software development but it is often manual and so expensive and error prone. One of the proposed solutions to this is to use model-based testing, in which testing is based on a model of how the system should behave. If the model has a formal semantics, then there is potential to automate systematic test generation. In this paper we consider the case where the semantics of the model is a set of refusal traces, also called failure traces. We show how the notions of fundamental refusal and fundamental refusal trace can be used to derive a normalised transition system, which we call an observation transition system (OTS), from the semantics. We then show how, if this OTS has finitely many states, and we are given a bound m, one can produce a corresponding complete test suite: one that is guaranteed to determine correctness as long as the number of states of the OTS defined by the semantics of the system under test has no more than m states. In practice, the choice of value for m might be based on domain knowledge or a cost-benefit analysis. As far as we are aware, this is the first work to show how a finite complete test suite can be derived when the semantics under consideration is a set of refusal traces.

Model-based diversity-driven learn-to-rank test case prioritization

Model-based Test Case Prioritization utilizing similarity metrics has proved effective in software testing. However, the utility of similarity metrics in it varies with test scenarios, hindering its universal effectiveness and performance optimization. To tackle this problem, we propose a Diversity-driven Learn-to-rank model-based TCP approach, named DLTCP, for optimizing early fault detection performance. Our method first employs the whale optimization algorithm to search for a suitable set of similarity metrics from a pool of existing candidates. This search process determines which metrics should be used. According to each selected metric, test cases are then prioritized. The resulting test case rankings are used as the training data for DLTCP. Finally, the proposed method incorporates random forest to train a ranking model for test case prioritization. As such, it can fuse multiple similarity metrics to improve the TCP performance. We conduct extensive experiments to evaluate our method’s performance using the average percentage fault detected (APFD) as metric. The experimental results show that DLTCP achieve an average APFD value of 0.953 for seven classic benchmark models, which is 11.37% higher than that of the state-of-the-art algorithms. It can well select a set of similarity metrics for effective fusion, demonstrating competitive performance in early fault detection.

Object Constraint Language based test case optimization with modified Average Percentage of Fault Detection metric

AbstractTesting is one of the most time‐consuming and unpredictable processes within the software development life cycle. As a result, many test case optimization (TCO) techniques have been proposed to make this process more scalable. Object Constraint Language (OCL) was initially introduced as a constraint language to provide additional details to Unified Modeling Language models. However, as OCL continues to evolve, an increasing number of systems are being expressed by this language. Despite this growth, a noticeable research gap exists for the testing of systems whose specifications are expressed in OCL. In our previous work, we verified the effectiveness and efficiency of performing the test case prioritization (TCP) process for these systems. In this study, we extend our previous work by integrating the test case minimization (TCM) process to determine whether TCM can also benefit the testing process under the context of OCL. The evaluation of TCO approaches often relies on well‐established metrics such as the average percentage of fault detection (APFD). However, the suitability of APFD for model‐based testing (MBT) is not ideal. This paper addresses this limitation by proposing a modification to the APFD metric to enhance its viability for MBT scenarios. We conducted four case studies to evaluate the feasibility of integrating the TCM and TCP processes into our proposed approach. In these studies, we applied the multi‐objective optimization algorithm NSGA‐II and the genetic algorithm independently to the TCM and TCP processes. The objective was to assess the effectiveness and efficiency of combining TCM and TCP in enhancing the testing phase. Through experimental analysis, the results highlight the benefits of integrating TCM and TCP in the context of OCL‐based testing, providing valuable insights for practitioners and researchers aiming to optimize their testing efforts. Specifically, the main contributions of this work include the following: (1) we introduce the integration of the TCM process into the TCO process for systems expressed by OCL. This integration benefits the testing process further by reducing redundant test cases while ensuring sufficient coverage. (2) We comprehensively analyze the limitations associated with the commonly used metric, APFD, and then, a modified version of the APFD metric has been proposed to overcome these weaknesses. (3). We systematically evaluate the effectiveness and efficiency of OCL‐based TCO processes on four real‐world case studies with different complexities.

Unifying frameworks for complete test strategies

The field of model-based testing has witnessed the development of several test strategies on finite state machines. Although these strategies are often related, little effort has been made to explicitly identify patterns shared between them, and their concrete implementations as well as completeness proofs regularly exhibit redundancy. In this paper, we propose an approach for the systematic verification and implementation of strategies for the language-equivalence conformance relation. We present frameworks in the form of higher order functions that implement shared behaviour once and encapsulate diverging behaviour in procedural parameters, thus reducing duplication and improving maintainability and extensibility. We show that this simplifies completeness proofs by proving complete all considered strategies using the same argument. All presented frameworks, proofs, and concrete strategy implementations have been mechanised using the proof assistant Isabelle.

Remembering the truth or falsity of advertising claims: A preregistered model-based test of three competing theoretical accounts

Given the large amount of information that people process daily, it is important to understand memory for the truth and falsity of information. The most prominent theoretical models in this regard are the Cartesian model and the Spinozan model. The former assumes that both “true” and “false” tags may be added to the memory representation of encoded information; the latter assumes that only falsity is tagged. In the present work, we contrasted these two models with an expectation-violation model hypothesizing that truth or falsity tags are assigned when expectations about truth or falsity must be revised in light of new information. An interesting implication of the expectation-violation model is that a context with predominantly false information leads to the tagging of truth whereas a context with predominantly true information leads to the tagging of falsity. To test the three theoretical models against each other, veracity expectations were manipulated between participants by varying the base rates of allegedly true and false advertising claims. Memory for the veracity of these claims was assessed using a model-based analysis. To increase methodological rigor and transparency in the specification of the measurement model, we preregistered, a priori, the details of the model-based analysis test. Despite a large sample size (N = 208), memory for truth and falsity did not differ, regardless of the base rates of true and false claims. The results thus support the Cartesian model and provide evidence against the Spinozan model and the expectation-violation model.

Adaptation to extreme weather events using pre-conditioning: a model-based testing of novel resilience algorithms on a residential case study

Climate change increases the frequency and intensity of extreme weather events that can be a prominent cause of power outages in North America. These events may cause buildings to experience outages for hours to days, endangering occupant well-being. Although typical adaptive strategies can offer assistance, they often demand substantial initial investments. Thus, due to the need for low-cost solutions, this paper evaluates the efficacy of the proposed pre-heating/cooling algorithm using smart thermostats. The ongoing research employs automated energy modelling through Python scripting to streamline the energy model upgrade process and the EnergyPlus Energy Management System (EMS) algorithm to incorporate pre-conditioning features during grid outages. The results indicated an average 18% improvement in peak intensity and 9% in overall performance during extreme events. Also, it offers the potential for future studies to employ this methodology in assessing the effects of other low-cost strategies for adapting to grid disruptions.

Assessing the testing skills transfer of model-based testing on testing skill acquisition

Assessing the testing skills transfer of model-based testing on testing skill acquisition

Exhaustive property oriented model-based testing with symbolic finite state machines

Exhaustive property oriented model-based testing with symbolic finite state machines

MTR Model-Based Testing Framework MTR Model-Based Testing Framework MTR Model-Based Testing Framework

In this article we propose a novel, free and open- source model-based testing framework for finite state machine specifications. The various model handling and test generation options make the framework suitable for testing complex systems and provide a solid background for investigating different automated test design methodologies. The complexity and fault detection capabilities of the available algorithms are investigated through analytical analyses and simulations applying randomly injected faults.

Effects of Peer Teaching Method in Software Testing Course in Comparison with Traditional Teaching Method

This action research project incorporated a peer teaching model for creating an active learning environment for teaching the course “Software Verification, Validation, and Testing”. Topics covered in the course included an overview of software testing, software quality, types of software testing, test case design techniques, test documentation, software test frameworks; model-based testing, software verification, and validation. The research aimed to explore to what extent peer teaching would increase students’ learning motivation, and engagement, develop critical thinking skills, and their performance. The study was conducted for one semester and data were generated using questionnaires, test scores, and student activity records as instruments. The test score data is collected from 10 students taking the course. The data were analyzed using qualitative (textual data obtained from open-ended responses) and quantitative (numerical data from questionnaires, and test scores) methods. The result showed that the peer teaching approach has a positive effect on the attitudes of students toward software testing courses compared to lecture mode instruction.

EFSM Model-Based Testing for Android Applications

Model-based testing provides an effective means for ensuring the quality of Android apps. Nevertheless, existing models that focus on event sequences and abstract them into Finite State Machines (FSMs) may lack precision and determinism because of the different data values of events that can result in various states of Android applications. To address this issue, a novel model based on Extended Finite State Machines (EFSMs) for Android apps is proposed in this paper. The approach leverages machine learning to infer data constraints on events and annotates them on state transitions, leading to a more precise and deterministic model. Additionally, a state abstraction strategy is presented to further refine the model. Besides, test diversity plays a vital role in enhancing test suite effectiveness. To achieve high coverage and fault detection, test cases are generated from the EFSM model with the help of a Genetic Algorithm (GA), guided by test diversity. To evaluate the effectiveness of our approach, this paper carries out experiments on 93 open-source apps. The results show that our approach performs better in code coverage and crash detection than the existing open-source model-based testing tools. Particularly, the 19 unique crashes that involve complex data constraints are detected by our approach.

CMBMeTest: Generation of Test Suites Using Model-Based Testing Plus Constraint Programming and Metamorphic Testing

Various software testing techniques have been shown to be successful in producing high-quality test suites for software where the code is not accessible (black-box approach). Nevertheless, no method has been found to guide combining some of these in a general way. In this study, a test suite generation method for black-box software called CMBMeTest was created to respond to these challenges. It employs several coupled software testing techniques, namely, model-based testing (MBT), constraint programming (CP), and metamorphic testing (MT). CMBMeTest provides step-by-step instructions for using the information available (such as program specifications, inputs and outputs) to create an initial test suite that covers the model obtained, using a combination of MBT and CP (referred to as MBT+CP). Furthermore, using the metamorphic relations (MRs) of MT, a better test suite was produced from that initial test suite. The method allows particular stages to be iterated to improve the results by building new models and new MRs. A comprehensive case study was conducted, employing CMBMeTest to produce encouraging results. Mutation testing was used to evaluate the test suite, and the first round produced a high mutation score. A more detailed model was used to repeat the process, with similar outcomes.

Should We Test the Model Assumptions Before Running a Model-based Test?

Statistical methods are based on model assumptions, and it is statistical folklore that a method's model assumptions should be checked before applying it. This can be formally done by running one or more misspecification tests testing model assumptions before running a method that requires these assumptions; here we focus on model-based tests. A combined test procedure can be defined by specifying a protocol in which first model assumptions are tested and then, conditionally on the outcome, a test is run that requires or does not require the tested assumptions. Although such an approach is often taken in practice, much of the literature that investigated this is surprisingly critical of it.Our aim is to explore conditions under which model checking is advisable or not advisable. For this, we review results regarding such ``combined procedures'' in the literature, we review and discuss controversial views on the role of model checking in statistics, and we present a general setup in which we can show that preliminary model checking is advantageous, which implies conditions for making model checking worthwhile.

Model-Based Testing Approaches using UML Diagrams: A Systematic Literature Review

Software Unit Testing (SUT) is the starting point for Model-Based Testing (MBT), a testing method. The Unified Modeling Language (UML) has become the standard for modelling software in professional and academic settings. There are various uses for the modelling language known as UML. The findings of an SLR on UML-based model-based testing methodologies are presented in this paper. Thirty-five primary articles about six research issues were examined using selection and exclusion criteria. Methods, model class, intermediate format use, and testing methodology are the primary points of examination. The review outcomes identify future research needs and avenues of inquiry.

Implementation of Model Based Testing for Testing Kawn Subscriptions Manager Application

This research focuses on the Kawn Subscriptions Manager app, which was developed to help Food and Beverage (F&B) businesses manage customer subscriptions. Currently, the Kawn Subscriptions Manager application does not have a standardized test suite that has been validated. Therefore, in this research, Model-Based Testing (MBT) is applied in testing the Kawn Subscriptions Manager application as a testing method to produce a specific and technical test suite. MBT is a testing method based on the behavioral model of the system being tested. Then, to measure the adequacy of test cases generated by MBT, mutation testing is used. The model is created using the GraphWalker tool and testing is done automatically using Selenium and testNG in the Eclipse IDE text editor. From the results of the study, it is known that MBT testing generates test cases from the behavior model of the function being tested. Testing with MBT shows that no failures were found in the tested functions. Then the average mutation score is 72.18% which indicates that the test cases generated by MBT are adequate or in the medium category. However, it is important to note that while MBT was successful in generating test cases that examined the application's behavior, it was found that the method was unable to detect failures caused by mutants that did not significantly affect the application's behavior. This finding suggests that while MBT can be an effective tool for testing key functions, additional methods are needed to identify failures that may result from minor or insignificant changes to the application code.

Modified-multi objective firefly optimization algorithm for object oriented applications test suites optimization

Model-based testing is a crucial but challenging stage of the software development process. The process of model-based testing needsto be optimized, which is a difficult task. In this article, we present an approach for selecting minimum test suites that is based on themeta-heuristic firefly algorithm. We modify the firefly algorithm and define the suitable multi-objective function to optimize the testsuites. The suggested approach uses firefly behavior to address the current issue. The modified approach chooses the best test suitesthat quickly find the maximum coverage in less time.

On Stochastic Optimization for Smartphone CPU Energy Consumption Decrease

Extending smartphone working time is an ongoing endeavour becoming more and more important with each passing year. It could be achieved by more advanced hardware or by introducing energy-aware practices to software, and the latter is a more accessible approach. As the CPU is one of the most power-hungry smartphone devices, Dynamic Voltage Frequency Scaling (DVFS) is a technique to adjust CPU frequency to the current computational needs, and different algorithms were already developed, both energy-aware and energy-agnostic kinds. Following our previous work on the subject, we propose a novel DVFS approach to use simultaneous perturbation stochastic approximation (SPSA) with two noisy observations for tracking the optimal frequency and implementing several algorithms based on it. Moreover, we also address an issue of hardware lag between a signal for the CPU to change frequency and its actual update. As Android OS could use a default task scheduler or an energy-aware one, which is capable of taking advantage of heterogeneous mobile CPU architectures such as ARM big.LITTLE, we also explore an integration scheme between the proposed algorithms and OS schedulers. A model-based testing methodology to compare the developed algorithms against existing ones is presented, and a test suite reflecting real-world use case scenarios is outlined. Our experiments show that the SPSA-based algorithm works well with EAS with a simplified integration scheme, showing CPU performance comparable to other energy-aware DVFS algorithms and a decreased energy consumption.