Testing Preorders Research Articles

Multiparty testing preorders

Variants of the must testing approach have been successfully applied in service oriented computing for analysing the compliance between (contracts exposed by) clients and servers or, more generally, between two peers. It has however been argued that multiparty scenarios call for more permissive notions of compliance because partners usually do not have full coordination capabilities. We propose two new testing preorders, which are obtained by restricting the set of potential observers. For the first preorder, called uncoordinated, we allow only sets of parallel observers that use different parts of the interface of a given service and have no possibility of intercommunication. For the second preorder, that we call individualistic, we instead rely on parallel observers that perceive as silent all the actions that are not in the interface of interest. We have that the uncoordinated preorder is coarser than the classical must testing preorder and finer than the individualistic one. We also provide a characterisation in terms of decorated traces for both preorders: the uncoordinated preorder is defined in terms of must-sets and Mazurkiewicz traces while the individualistic one is described in terms of classes of filtered traces that only contain designated visible actions and must-sets.

Full-abstraction for client testing preorders

• After a general introduction we recall the basic notion and existing results (Section 2). • In Section 3 we show an inductive characterisation of usable clients wrt must testing. • In Section 4 we give a coinductive characterisation of the Must client preorder. • Our characterisation us present the first decision procedure for this preorder. • After a brief digression about an expressiveness result, we characterise the compliance preorder for clients (Section 6). • Our results depend on auxiliary sets sharing order-theoretic justifications, suggesting a way to study testing preorders. Client testing preorders relate tests (clients) instead of processes (servers), and are usually defined using either must testing or a compliance relation. Existing characterisations of these preorders are unsatisfactory for they rely on the notion of usable clients which, in turn, are defined using an existential quantification over the servers that ensure client satisfaction. In this paper we characterise the set of usable clients wrt must testing for finite-branching LTSs, and give a sound and complete decision procedure for it. We also provide novel coinductive characterisations of the client preorders due to must and compliance, which we use to show that these preorders are decidable, thus positively answering the question opened in [5] , [3] .

Testing Preorders for dMTS

Testing preorders on component specifications ensure that replacing a specification by a refined one does not introduce unwanted behavior in an overall system. Considering deadlocks as unwanted, the preorder can be characterized by a failure semantics on Labeled Transition Systems (LTSs). In previous work, we have generalized this to Modal Transition Systems (MTSs) with a new, MTS-specific testing idea. In the present article, we generalize this idea further to DMTS, a subclass of disjunctive MTSs. On the one hand, the testing preorder can be characterized by the same failure semantics, and dMTS have no additional expressivity in our setting. On the other hand, the technical treatment is significantly harder and, surprisingly, the preorder is not compositional. Furthermore, we regard deadlocks and divergence (infinite unobservable runs) as unwanted and characterize the testing preorder with an unusual failure-divergence semantics. This preorder is already on LTSs strictly coarser—and hence arguably better—than the traditional failure-divergence preorder. It is a precongruence on dMTS, also for hiding, and much easier to handle than the deadlock-based preorder. It arises as well from a new variant of De Nicola’s and Hennessy’s must-testing.

Basic behavioral models for software product lines: Expressiveness and testing pre-orders

In order to provide a rigorous foundation for Software Product Lines (SPLs), several fundamental approaches have been proposed to their formal behavioral modeling. In this paper, we provide a structured overview of those formalisms based on labeled transition systems and compare their expressiveness in terms of the set of products they can specify. Moreover, we define the notion of tests for each of these formalisms and show that our notions of testing precisely capture product derivation, i.e., all valid products will pass the set of test cases of the product line and each invalid product fails at least one test case of the product line.

Characterisations of testing preorders for a finite probabilistic π -calculus

Abstract We consider two characterisations of the may and must testing preorders for a probabilistic extension of the finite π -calculus: one based on notions of probabilistic weak simulations, and the other on a probabilistic extension of a fragment of Milner–Parrow–Walker modal logic for the π -calculus. We base our notions of simulations on similar concepts used in previous work for probabilistic CSP. However, unlike the case with CSP (or other non-value-passing calculi), there are several possible definitions of simulation for the probabilistic π -calculus, which arise from different ways of scoping the name quantification. We show that in order to capture the testing preorders, one needs to use the “earliest” simulation relation (in analogy to the notion of early (bi)simulation in the non-probabilistic case). The key ideas in both characterisations are the notion of a “characteristic formula” of a probabilistic process, and the notion of a “characteristic test” for a formula. As in an earlier work on testing equivalence for the π -calculus by Boreale and De Nicola, we extend the language of the π -calculus with a mismatch operator, without which the formulation of a characteristic test will not be possible.

Probabilistic may/must testing: retaining probabilities by restricted schedulers

Abstract This paper considers the probabilistic may/must testing theory for processes having external, internal, and probabilistic choices. We observe that the underlying testing equivalence is too strong and distinguishes between processes that are observationally equivalent. The problem arises from the observation that the classical compose-and-schedule approach yields unrealistic overestimation of the probabilities, a phenomenon that has been recently well studied from the point of view of compositionality, in the context of randomized protocols and in probabilistic model checking. To that end, we propose a new testing theory, aiming at preserving the probability information in a parallel context. The resulting testing equivalence is insensitive to the exact moment the internal and the probabilistic choices occur. We also give an alternative characterization of the testing preorder as a probabilistic ready-trace preorder.

A precongruence format for should testing preorder

Should testing preorder was proposed as a liveness-preserving precongruence for a process algebra except nondeterministic choice operator [22]. However, this precongruence result cannot be generalized to other languages before carefully proved. In the paper, we handle this by showing that τdes format is a precongruence format for should testing preorder. A precongruence format guarantees the precongruence of given preorder by imposing syntactic restrictions on languages. The τdes format [23] was suggested to be a precongruence format for testing preorder, which is incomparable with should testing preorder on discriminative power. Moreover, we give its applications by looking into the ACP language.

A theory of contracts for Web services

Contracts are behavioral descriptions of Web services. We devise a theory of contracts that formalizes the compatibility of a client with a service, and the safe replacement of a service with another service. The use of contracts statically ensures the successful completion of every possible interaction between compatible clients and services. The technical device that underlies the theory is the filter , which is an explicit coercion preventing some possible behaviors of services and, in doing so, make services compatible with different usage scenarios. We show that filters can be seen as proofs of a sound and complete subcontracting deduction system which simultaneously refines and extends Hennessy's classical axiomatization of the must testing preorder. The relation is decidable, and the decision algorithm is obtained via a cut-elimination process that proves the coherence of subcontracting as a logical system. Despite the richness of the technical development, the resulting approach is based on simple ideas and basic intuitions. Remarkably, its application is mostly independent of the language used to program the services or the clients. We outline the practical aspects of our theory by studying two different concrete syntaxes for contracts and applying each of them to Web services languages. We also explore implementation issues of filters and discuss the perspectives of future research this work opens.

A theory of contracts for strong service compliance

We investigate, in a process algebraic setting, a new notion of correctness for service compositions, which we callstrong service compliance: composed services are strong compliant if their composition is both deadlock and livelock free (this is the traditional notion of compliance), and whenever a message can be sent to invoke a service, it is guranteed to be ready to serve the invocation. We also define a new notion of refinement, calledstrong subcontract pre-order, suitable for strong compliance: given a composition of strong compliant services, we can replace any service with any other service in subcontract relation while preserving the overall strong compliance. Finally, we present a characterisation of the strong subcontract pre-order by resorting to the theory of a (should) testing pre-order.

Characterising Testing Preorders for Finite Probabilistic Processes

In 1992 Wang & Larsen extended the may- and must preorders of De Nicola and Hennessy to processes featuring probabilistic as well as nondeterministic choice. They concluded with two problems that have remained open throughout the years, namely to find complete axiomatisations and alternative characterisations for these preorders. This paper solves both problems for finite processes with silent moves. It characterises the may preorder in terms of simulation, and the must preorder in terms of failure simulation. It also gives a characterisation of both preorders using a modal logic. Finally it axiomatises both preorders over a probabilistic version of CSP.

A theory of contracts for web services

Contracts are behavioural descriptions of Web services. We devise a theory of contracts that formalises the compatibility of a client to a service, and the safe replacement of a service with another service. The use of contracts statically ensures the successful completion of every possible interaction between compatible clients and services. The technical device that underlies the theory is the definition of filters, which are explicit coercions that prevent some possible behaviours of services and, in doing so, they make services compatible with different usage scenarios. We show that filters can be seen as proofs of a sound and complete subcontracting deduction system which simultaneously refines and extends Hennessy's classical axiomatisation of the must testing preorder. The relation is decidable and the decision algorithm is obtained via a cut-elimination process that proves the coherence of subcontracting as a logical system. Despite the richness of the technical development, the resulting approach is based on simple ideas and basic intuitions. Remarkably, its application is mostly independent of the language used to program the services or the clients. We also outline the possible practical impact of such a work and the perspectives of future research it opens.

CONFLICTS AND FAIR TESTING

This paper studies conflicts from a process-algebraic point of view and shows how they are related to the testing theory of fair testing. Conflicts have been introduced in the context of discrete event systems, where two concurrent systems are said to be in conflict if they can get trapped in a situation where they are waiting or running endlessly, forever unable to complete their common task. In order to analyse complex discrete event systems, conflict-preserving notions of refinement and equivalence are needed. This paper characterises an appropriate refinement, called the conflict preorder, and provides a denotational semantics for it. Its relationship to other known process preorders is explored, and it is shown to generalise the fair testing preorder in process-algebra for reasoning about conflicts in discrete event systems.

On testing UML statecharts

We present a formal framework for notions related to testing and model based test generation for a behavioural subset of UML Statecharts (UMLSCs). This framework builds, on one hand, upon formal testing and conformance theory that has originally been developed in the context of process algebras and Labeled Transition Systems (LTSs), and, on the other hand, upon our previous work on formal semantics for UMLSCs. The paper covers the development of proper extensional testing preorders and equivalence for UMLSCs. We present an algorithm for testing equivalence verification which is based on an intensional characterization of the testing relations. Testing equivalence verification is reduced to bisimulation equivalence verification. We also address the issue of conformance testing and present a formal conformance relation together with a test case generation algorithm which is proved sound and exhaustive w.r.t. the conformance relation. We show results on the formal relationship of the testing relations with the conformance one. The comprehensive and uniform approach presented in this paper sets the theoretical basis for UMLSCs testing frameworks and makes them available for practitioners in industry where the UML has become a de facto standard, in particular there where it is used for the development of complex concurrent systems.

Measuring the performance of asynchronous systems with PAFAS

Based on Process Algebra for Faster Asynchronous Systems (PAFAS), a testing-based faster-than relation has previously been developed that compares the worst-case efficiency of asynchronous systems. This approach reveals that pipelining does not improve efficiency in general; that it does so in practice depends on assumptions about the user behaviour. As a case study for testing under such assumptions, we adapt the PAFAS-approach to a setting where user behaviour is known to belong to a specific, but often occurring class of request–response behaviours. Just as the testing preorder in classical testing, the original faster-than relation is qualitative. We give it a quantitative reformulation for the general approach; based on this, we demonstrate in our case study how to determine an asymptotic performance measure for finite-state processes. With this result, we can show that pipelining indeed improves efficiency in our setting, and we discuss additional examples.

Domain theory, testing and simulation for labelled Markov processes

This paper presents a fundamental study of similarity and bisimilarity for labelled Markov processes (LMPs). The main results characterize similarity as a testing preorder and bisimilarity as a testing equivalence. In general, LMPs are not required to satisfy a finite-branching condition—indeed the state space may be a continuum, with the transitions given by arbitrary probability measures. Nevertheless we show that to characterize bisimilarity it suffices to use finitely-branching labelled trees as tests. Our results involve an interaction between domain theory and measure theory. One of the main technical contributions is to show that a final object in a suitable category of LMPs can be constructed by solving a domain equation D ≅ V ( D ) Act , where V is the probabilistic powerdomain. Given an LMP whose state space is an analytic space, bisimilarity arises as the kernel of the unique map to the final LMP. We also show that the metric for approximate bisimilarity introduced by Desharnais, Gupta, Jagadeesan and Panangaden generates the Lawson topology on the domain D .

Comparing the worst-case efficiency of asynchronous systems with PAFAS

A timed CCS-like process algebra PAFAS and a testing scenario are developed for evaluating the temporal worst-case efficiency of asynchronous concurrent systems. Each action is associated with a maximal time delay, which allows components to work with arbitrary relative speeds; for simplicity, the maximal delay is 1 or 0, but time is continuous. The canonical testing preorder associated to our timed testing compares worst-case efficiency; we show that this efficiency preorder can equivalently be defined considering only discrete time, which is of course much simpler. Then we characterize the efficiency preorder with some kind of refusal traces; despite the rather weak control an asynchronous test environment has, this gives quite detailed insight into the temporal system behaviour. Since the preorder is not a precongruence for choice, we refine it to the efficiency precongruence, which is a precongruence for all operators of the algebra including recursion.

Testing preorders for probabilistic processes can be characterized by simulations

Transition systems are well established as a semantic model for distributed systems. There are several preorders that serve as criteria for refinement of an abstract transition system to a more concrete one. To reason about probabilistic phenomena such as failures and randomization, we need to extend models and methods that have proven successful for nonprobabilistic systems to a probabilistic setting. In this paper, we develop a refinement preorder for a probabilistic extension of the transition systems model. The preorder is based on a notion of testing, where refinement corresponds to an improvement in the “worst-case” behavior of a process. The main result of the paper is that this preorder can be described by a notion of probabilistic simulation, which generalizes the standard simulation preorder for ordinary transition system. To our knowledge, this simulation preorder has not been previously described in the literature, and is strictly weaker than previously proposed simulations for probabilistic transition systems.

Fast asynchronous systems in dense time

A testing scenario in the sense of De Nicola and Hennessy is developed to measure the worst-case efficiency of asynchronous systems using dense time. For all three variants considered, it is shown that one can equivalently use discrete time; in the discrete versions, one variant coincides with an approach based on discrete time in Vogler (CONCUR95, Lecture Notes in Computer Science, vol. 962, Springer, Berlin, 1995, pp. 299–312), and thus we can clarify the assumptions behind this approach. The resulting testing-preorders are characterized with some kind of refusal traces and shown to satisfy some properties that make them attractive as faster-than relations. The three testing preorders are incomparable in general, but for some interesting classes of systems implications are shown.

Finite axiom systems for testing preorder and De Simone process languages

We prove that testing preorder of De Nicola and Hennessy is preserved by all operators of De Simone process languages. Building upon this result we propose an algorithm for generating axiomatisations of testing preorder for arbitrary De Simone process languages. The axiom systems produced by our algorithm are finite and complete for processes with finite behaviour. In order to achieve completeness for a subclass of processes with infinite behaviour we use one infinitary induction rule. The usefulness of our results is illustrated in specification and verification of small concurrent systems, where suspension, resumption and alternation of execution of component systems occur. We argue that better specifications can be written in customised De Simone process languages, which contain both the standard operators as well as new De Simone operators that are specifically tailored for the task in hand. Moreover, the automatically generated axiom systems for such specification languages make the verification more straightforward.

Testing Preorders for Probabilistic Processes

We present a testing preorder for probabilistic processes based on a quantification of the probability with which processes pass tests. The theory enjoys close connections with the classical testing theory of De Nicola and Hennessy in that whenever a process passes a test with probability 1 (respectively some nonzero probability) in our setting, then the process must (respectively may ) pass the test in the classical theory. We also develop an alternative characterization of the probabilistic testing preorders that takes the form of a mapping from probabilistic traces to the interval [0, 1], where a probabilistic trace is an alternating sequence of actions and probability distributions over actions. Finally, we give proof techniques , derived from the alternative characterizations, for establishing preorder relationships between probabilistic processes. The utility of these techniques is demonstrated by means of some simple examples.