Conformance Test Tool Research Articles

Analysis of Flatness and Alignment in the Design of Wireless X-Ray Conformance Test Tool.

Analysis of Flatness and Alignment in the Design of Wireless X-Ray Conformance Test Tool.

FlowIO: Flow cytometry standard conformance testing, editing, and export tool.

The Flow Cytometry Standard (FCS) format is a widely accepted norm for storing Flow Cytometry (FCM) data. Its goal as a standard is to allow FCM data sharing and re-analysis. Over more than three decades of its existence FCS has evolved into a well-defined, flexible file format reflecting technical changes in the FCM field. Its flexibility as well as rising numbers of instrument vendors leads to suboptimal implementations of FCS in some cases. Such situations compromise the primary goal of the standard and hinder the ability to reproduce FCM analyses. It is further underlined by rapid rise of advanced FCM analyses, often carried out outside traditional software tools and heavily relying on standard data storage and presentation. We have developed flowIO, an R package which tests FCS file conformance with the standard as defined by International Society for Advancement of Cytometry (ISAC) normative. Along with the package we provide a web based application (also at http://bioinformin.cesnet.cz/flowIO/) allowing user friendly access to the conformance testing as well as FCS file editing and export for further analysis.

Quantifying conformance using the Skorokhod metric

The conformance testing problem for dynamical systems asks, given two dynamical models (e.g., as Simulink diagrams), whether their behaviors are “close” to each other. In the semi-formal approach to conformance testing, the two systems are simulated on a large set of tests, and a metric, defined on pairs of real-valued, real-timed trajectories, is used to determine a lower bound on the distance. We show how the Skorokhod metric on continuous dynamical systems can be used as the foundation for conformance testing of complex dynamical models. The Skorokhod metric allows for both state value mismatches and timing distortions, and is thus well suited for checking conformance between idealized models of dynamical systems and their implementations. We demonstrate the robustness of the metric by proving a transference theorem: trajectories close under the Skorokhod metric satisfy “close” logical properties in the timed linear time logic FLTL (Freeze LTL) containing a rich class of temporal and spatial constraint predicates involving time and value freeze variables. We provide efficient window-based streaming algorithms to compute the Skorokhod metric for both piecewise affine and piecewise constant traces, and use these as a basis for a conformance testing tool for Simulink. We experimentally demonstrate the effectiveness of our tool in finding discrepant behaviors on a set of control system benchmarks, including an industrial challenge problem.

Advancing HL7 v2 to New Heights: A Platform for Developing Specifications, Test Plans, and Testing Tools

Development of HL7 v2 data exchange interface specifications has long been problematic, plagued with ambiguous and inconsistent requirement specifications. This situation leads to potential misinterpretation by implementers, thus limiting the effectiveness of the specification and creating artificial and unnecessary barriers to interoperability. Likewise, the ability to test implementations effectively for conformance to the specifications is hindered. The current approach of specification development and test plan creation relies on word processing tools, meaning implementers and testers must read and interpret the information in these documents and then translate it into machine-computable requirements and test assertions. This approach is error prone—a better methodology is needed. We present a set of productivity tools in an integrated platform that allow users to define and constrain HL7 v2 specifications and to develop test plans that result in machine-computable artifacts. A testing infrastructure and framework subsequently uses these artifacts to create conformance testing tools automatically. We present and demonstrate the utility of a platform for developing specifications, writing test plans, and creating testing tools. The value proposition of this end-to-end methodology is explained for authors writing HL7 v2 specifications, for developers implementing interfaces, and for testers creating validation tools.

Enhancing Conformance Testing Using Symbolic Execution for Network Protocols

Security protocols are notoriously difficult to get right, and most go through several iterations before their hidden security vulnerabilities, which are hard to detect, are triggered. To help protocol designers and developers efficiently find non-trivial bugs, we introduce SYMCONF, a practical conformance testing tool that generates high-coverage test input packets using a conformance test suite and symbolic execution. Our approach can be viewed as the combination of conformance testing and symbolic execution: 1) it first selects symbolic inputs from an existing conformance test suite; 2) it then symbolically executes a network protocol implementation with the symbolic inputs; and 3) it finally generates high-coverage test input packets using a conformance test suite. We demonstrate the feasibility of this methodology by applying SYMCONF to the generation of a stream of high quality test input packets for multiple implementations of two network protocols, the Kerberos Telnet protocol and Dynamic Host Configuration Protocol (DHCP), and discovering non-trivial security bugs in the protocols.

A conformance test tool for next generation network applications and systems

AbstractThis paper describes the architecture and operations of a test system NGNAS‐SIOTP. NGNAS‐SIOTP is designed based on the Testing and Test Control Notation Version 3 (TTCN‐3) specifications for the conformance testing of next generation network applications and systems. By specifying conformance test cases in abstract level, the test cases are language and platform‐independent, and the implementation of the test cases can be more convenient. We use an Open Mobile Alliance (OMA) Browsing test case of XHTML Structure Module and an IP Multimedia Subsystem (IMS) test case of Replaces Extension Header Field to illustrate how conformance tests can be implemented and conducted in our NGNAS‐SIOTP. These test cases conform to the OMA Enabler Test Specification (Conformance) for OMA Browsing and RFC documents for IMS. Copyright © 2010 John Wiley & Sons, Ltd.

ITRI‐WiMAXT: A WiMAX conformance testing tool

AbstractThe WiMAX Forum certifies and promotes the compatibility and interoperability of broadband wireless products where many testcases need to be developed for WiMAX procedures. In this paper, we develop a conformance test tool called ITRI‐WiMAXT based on the TTCN‐3 specifications. Then we show how WiMAX procedures are tested in ITRI‐WiMAXT. Based on ITRI‐WiMAXT, we have proposed several test procedures accepted by the WiMAX Forum. Copyright © 2009 John Wiley & Sons, Ltd.

Development of verification and conformance testing tools for a railway signaling communication protocol

Verification and conformance testing for protocol specification, the key part of the protocol development process, are complementary technologies employed to increase confidence that a system will function as stated in its specifications. In this paper, we verify the safety and liveness of the protocol specified for the Labeled Transition System (LTS) by using a model-checking method and implementing the testing tool, which experimentally demonstrates the presence of deadlock and reachability from the initial state to a random state. Implementing the testing tool can use modal mu-calculus to assess whether protocol model properties, presented by modal logic, meet protocol specifications. In addition, we propose a conformance testing tool to check correct implementation of sequences that have been derived by the UIO method from the specification of the protocol being verified. This generating tool uses the C++ language in the Microsoft Windows NT environment.

NTP-DownloadT: a conformance test tool for secured mobile download services

Many mobile data services have been developed. Before commercial launch of a data service, it is required to ensure that the service is implemented securely and correctly on a mobile device. To address this issue, Open Mobile Alliance (OMA) has defined test specifications to guarantee the conformance of data services on mobile devices. This paper describes a conformance test tool called NTP-DownloadT for OMA Download services. We discuss OMA Download requirements such as content discovery, authentication, delivery negotiation, content delivery and delivery confirmation. In this paper, we demonstrate how OMA Download test suite can be implemented to realise conformance tests.

NTP‐PoCT: a conformance test tool for push‐to‐talk over cellular network

AbstractThis paper describes a conformance test tool for push‐to‐talk over cellular network developed on an Open Mobile Alliance Service Interoperability Test Platform. Based on the TTCN‐3 specifications, we show how PoC test cases can be efficiently implemented on the test platform. Copyright © 2007 John Wiley & Sons, Ltd.

Business alignment: using process mining as a tool for Delta analysis and conformance testing

Increasingly, business processes are being controlled and/or monitored by information systems. As a result, many business processes leave their “footprints” in transactional information systems, i.e., business events are recorded in so-called event logs. Process mining aims at improving this by providing techniques and tools for discovering process, control, data, organizational, and social structures from event logs, i.e., the basic idea of process mining is to diagnose business processes by mining event logs for knowledge. In this paper we focus on the potential use of process mining for measuring business alignment, i.e., comparing the real behavior of an information system or its users with the intended or expected behavior. We identify two ways to create and/or maintain the fit between business processes and supporting information systems: Delta analysis and conformance testing. Delta analysis compares the discovered model (i.e., an abstraction derived from the actual process) with some predefined processes model (e.g., the workflow model or reference model used to configure the system). Conformance testing attempts to quantify the “fit” between the event log and some predefined processes model. In this paper, we show that Delta analysis and conformance testing can be used to analyze business alignment as long as the actual events are logged and users have some control over the process.

EbXML 표준 적합성 및 상호운용성 테스트 도구의 설계 및 구현

As of an international standard in e-business domains, ebXML solutions are required to be tested and authenticated whether they can satisfy the conformance and interoperability requirements. In this paper, we design and implement the ebXML conformance and interoperability testing tools that observe the standard specification in conformity with ebXML Message Service v2.0, ebXML Registry Service v2.0 and ebXML CPP/A Y2.0. With this regard, our research would serve in building the verification and authentication service infrastructure for fitting standard ebXML e-business solutions with ebXML conformance and interoperability testing tools.

On-the-fly conformance testing using SPIN

In this paper we report on the construction of a tool for conformance testing based on Spin. The Spin tool has been adapted such that it can derive the building blocks for constructing test cases, called test primitives, from systems described in Promela. The test primitives support the on-the-fly conformance testing process. Traditional derivation of tests from formal specifications suffers from the state-space explosion problem. Spin is one of the most advanced model checkers with respect to handling large state spaces. This advantage of Spin has been used for the derivation of test primitives from a Promela description. To reduce the state space, we introduce the on-the-fly testing framework. One of the components within this framework is the Primer. The Primer is responsible for deriving test primitives from a model of a system according to a well-defined and complete testing theory. Algorithms are presented which enable us to derive test primitives from a Promela description. These algorithms have been implemented in the adapted version of the Spin tool which acts as the Primer in the framework. Promising experiments have been carried out on an example case study. As a result of this study it is concluded that it is possible to derive test primitives automatically from Promela descriptions, construct test cases from these test primitives, and execute the test cases on-the-fly.

Conformance testing of X-25 packet level

This paper explains a method and tool for conformance testing of X.25 packet level protocol, and DTE equipment based on X.25 protocols. The method uses standard test suites, and standardized TTCN (Tree and Tabular Combined Notation).

OSI Conformance testing for bibliographic applications

The objective of OSI conformance testing is to provide a standardized and rigorous means of determining the extent to which a specific implementation conforms to an OSI protocol specification. As part of its ILL Implementation Program, the National Library of Canada has developed a test system and service for the interlibrary loan (ILL) protocol using the ISO standard methodology for conformance testing (ISO 9656). Development of test suites, conformance testing tools, and conformance testing services are presented. The article also discusses related topics such as interoperability testing, arbitration testing, and international harmonization of conformance testing.