Protocol State Machine Research Articles

ProInfer: inference of binary protocol keywords based on probabilistic statistics

Abstract Protocol reverse engineering is crucial in normative verification, and malware behavior analysis and vulnerability discovery. However, uncovering the structural features of binary protocols concealed within dense data representations remains a significant challenge. Accurately identifying keyword segments associated with message types is a prerequisite for meaningful semantic analysis and protocol state machine reduction. In this work, we introduce a novel approach for inferring keywords from binary protocols based on probabilistic statistics. Our method in terms of Byte employs heuristic rules to filter offset positions that are clearly unrelated to message types. We further filter candidate Byte-offsets utilizing constraint relations and provide the probabilistic ranking of each offset as the keyword segment. To enhance the reliability of keyword segment inference, we utilize the Monte Carlo algorithm to assess the difference between message clustering with candidate Byte-offset and random message clustering, and reorder candidate offsets according to the results. Then we can observe optimal values from both orderings and present the ultimate inference results. Experimental results demonstrate that our method excels in the accuracy of keyword segments identification compared with previous techniques.

Model based component development and analysis with ComMA

The lack of explicit and precise specifications of software interfaces between components often leads to integration issues during development and maintenance. To address this, we have developed a framework named ComMA (Component Modeling and Analysis) that supports model-based engineering of high-tech systems by precisely defining components and their interfaces. The framework is a family of Domain Specific Languages (DSLs) for modeling component interfaces, protocol state machines, time and data constraints, and constraints on relations between events of multiple interfaces. From these models a number of artifacts can be generated automatically to support analysis and various engineering tasks. ComMA has been developed in close collaboration with the Philips IGT business unit that develops minimally-invasive X-ray systems. This paper presents the experience we gained in creating the ComMA framework and its application in industrial practice. We describe and reflect on the technical, organizational and process-related aspects of deploying a non-trivial MDE solution in an industrial setting.

StateAFL: Greybox fuzzing for stateful network servers

Fuzzing network servers is a technical challenge, since the behavior of the target server depends on its state over a sequence of multiple messages. Existing solutions are costly and difficult to use, as they rely on manually-customized artifacts such as protocol models, protocol parsers, and learning frameworks. The aim of this work is to develop a greybox fuzzer (StateAFL) for network servers that only relies on lightweight analysis of the target program, with no manual customization, in a similar way to what the AFL fuzzer achieved for stateless programs. The proposed fuzzer instruments the target server at compile-time, to insert probes on memory allocations and network I/O operations. At run-time, it infers the current protocol state of the target server by taking snapshots of long-lived memory areas, and by applying a fuzzy hashing algorithm (Locality-Sensitive Hashing) to map memory contents to a unique state identifier. The fuzzer incrementally builds a protocol state machine for guiding fuzzing. We implemented and released StateAFL as open-source software. As a basis for reproducible experimentation, we integrated StateAFL with a large set of network servers for popular protocols, with no manual customization to accomodate for the protocol. The experimental results show that the fuzzer can be applied with no manual customization on a large set of network servers for popular protocols, and that it can achieve comparable, or even better code coverage and bug detection than customized fuzzing. Moreover, our qualitative analysis shows that states inferred from memory better reflect the server behavior than only using response codes from messages.

A progressive learning method on unknown protocol behaviors

Reverse analyzing of unknown protocol behaviors keeps being a tough nut in Protocol Reverse Engineering (PRE), which infers specifications of unknown protocols by observable information, especially when only transmitted messages are available. This paper proposes a novel protocol state machine model Stochastic Protocol finite-state Transducer (SPT) to describe the message interaction rules between communicating terminals in a probabilistic way attempting to simulate behavior rules of unknown protocols in certain implementation. Together with a state related field recognition and compensation method, a progressive SPT learning algorithm of unknown protocols named Sptia-PL, is designed and implemented to reconstruct the SPT of target protocol with the ability to predict succeeding behaviors. By updating the SPT progressively, the proposed method is able to learn continuously in linear time and remain the established model in optimal condition during the whole learning process. This strategy thoroughly avoids the state explosion problem existing in most state machine learning methods of PRE. Experiments on two open and three local collected datasets of FTP, SMTP and POP3 prove the rationality of SPT model and effectiveness of Sptia-PL algorithm by an average Accuracy over 0.94 and a Coverage close to 0.99. The small computing cost O(N) of this method and high confidence of results outperforms all the known state-of-the-art methods significantly.

Protocol State Machine Reverse Engineering with a Teaching-Learning Approach

In this work, we propose a novel solution to the problem of inferring the state machine of an unknown protocol. We extend and improve prior results on inferring Mealy machines, and present a new algorithm that accesses and interacts with a networked system that runs the unknown protocol in order to infer the Mealy machine representing the protocol's state machine. To demonstrate the viability of our approach, we provide an implementation and illustrate the operation of our algorithm on a simple example protocol, as well as on two real-world protocols, Modbus and MQTT.

Research on Analytical Method of Private Power Protocol

Based on the study of protocol reverse engineering, this paper proposes a method of deep message parsing for private power protocols. It can parse industrial control protocols, extract protocol fields and state machines without knowing protocol specifications. This method can be used to monitor the flow of private protocol network safely, and it is of great significance to ensure the safe and stable operation of power grid.

Wireless Anomaly Detection Based on IEEE 802.11 Behavior Analysis

Wireless communication networks are pervading every aspect of our lives due to their fast, easy, and inexpensive deployment. They are becoming ubiquitous and have been widely used to transfer critical information, such as banking accounts, credit cards, e-mails, and social network credentials. The more pervasive the wireless technology is going to be, the more important its security issue will be. Whereas the current security protocols for wireless networks have addressed the privacy and confidentiality issues, there are unaddressed vulnerabilities threatening their availability and integrity (e.g., denial of service, session hijacking, and MAC address spoofing attacks). In this paper, we describe an anomaly based intrusion detection system for the IEEE 802.11 wireless networks based on behavioral analysis to detect deviations from normal behaviors that are triggered by wireless network attacks. Our anomaly behavior analysis of the 802.11 protocols is based on monitoring the n-consecutive transitions of the protocol state machine. We apply sequential machine learning techniques to model the n-transition patterns in the protocol and characterize the probabilities of these transitions being normal. We have implemented several experiments to evaluate our system performance. By cross validating the system over two different wireless channels, we have achieved a low false alarm rate (<0.1%). We have also evaluated our approach against an attack library of known wireless attacks and has achieved more than 99% detection rate.

Automated transformations from UML behavior models to contracts

In model driven architecture (MDA), system requirements are first captured by UML (unified modeling language) use cases with sequence diagrams to describe their intended use and implemented by classes of objected-oriented languages in the subsequent design stages. It is important that the dynamic behavior specified by the sequence diagrams is in full compliance with the implementation classes. This paper proposes an automatic approach and tool support for generating class contracts, which define a precondition and a postcondition for each operation of the class. The former serves as a guard to ensure invocations of the operations respect the semantics introduced by the sequence diagrams, and the latter places the system in a legal state to facilitate the succeeding operation calls. The contracts can be easily mapped to code of an object-oriented language such as Java. Thus, the approach helps to bridge the gap between the requirements and design stages of system development process. We use our model transformation tool to first generate a UML protocol state machine from the sequence diagrams, and then derive the contracts for a controller class. The transformations take into account the concurrency and critical constructs of the respective UML diagrams.

An Automatic Network Protocol State Machine Inference Method in Protocol Reverse Engineering

To infer the network protocol state machine is very useful in network security-related contexts, both in research and management. This process follows an extension of the classic Angluins L* algorithm and has achieved an extended version of some Mealy automata to represent or model a communication protocol. The algorithm has been validated by inferring the protocol state machine from SMTPFTP protocol, and tested offline algorithms for the comparison experiments. The experimental results show that this method can more accurately identify the network protocol state machine and is of the important application value.

Protocol state machine reverse method based on labeling state

Protocol state machine can describe the behavior of a protocol,which can help to understand the behavior logic of protocol. Oriented towards text protocols,a statistical method was firstly used to extract the semantic keyword of representative message type,and an adjacency matrix was used to describe the sequential relationship between the message types,based on which the protocol states were labeled and a state transition diagram was built. The experimental results show that the method can accurately describe the sequential relationship between the message types and abstract state machine model accurately.

Position-based automatic reverse engineering of network protocols

Automatic protocol reverse engineering is a process of extracting protocol message formats and protocol state machine without access to the specification of target protocol. Protocol reverse engineering is useful for addressing many problems of network management and security, such as network management, honey-pot systems, intrusion detection, Botnet detection and prevention, and so on. Currently, protocol reverse engineering is mainly a manual and painstaking process which is time-consuming and error-prone. In this paper, we present a novel approach for automatic reverse engineering application-layer network protocols. We extract protocol keywords from network traces based on their support rates and variances of positions, reconstruct message formats, and infer protocol state machines. We implement our approach in a prototype system called AutoReEngine and evaluate it over four text-based protocols (HTTP, POP3, SMTP and FTP) and two binary protocols (DNS and NetBIOS). The results show that our AutoReEngine outperforms the existing algorithms.

A study on the scalable flow model of web services choreography and orchestration based on dynamic workflow

The goal of choreography is to define accurately the sequence of interactions between cooperating web services and guarantee that changes of participating work flow still follow the definition of message exchange. According to the requirement of the municipal and the district petition business docking, and considering the characteristics of the dynamic workflow of petition letters processing and the interaction failure easily caused in the loose coupling environment of inter-process interaction and cooperation mismatches resulting from the participating resources autonomy, this article proposes a method of services choreography and orchestration based on the interaction protocol state machine, which controls the automatic advancement of the petition interaction protocols sequence according to the interaction rules. Ten months operation of this method in the municipal and the district petition businesses docking platform shows that it helps to shield the differences of different dynamic processes, tolerate interaction failures, and accordingly reduce the maintenance complexity.

Synthesizing Distributed Protocol Specifications from a UML State Machine Modeled Service Specification

The object-oriented paradigm is widely applied in designing and implementing communication systems. Unified Modeling Language (UML) is a standard language used to model the design of object-oriented systems. A protocol state machine is a UML adopted diagram that is widely used in designing communication protocols. It has two key attractive advantages over traditional finite state machines: modeling concurrency and modeling nested hierarchical states. In a distributed communication system, each entity of the system has its own protocol that defines when and how the entity exchanges messages with other communicating entities in the system. The order of the exchanged messages must conform to the overall service specifications of the system. In object-oriented systems, both the service and the protocol specifications are modeled in UML protocol state machines. Protocol specification synthesis methods have to be applied to automatically derive the protocol specification from the service specification. Otherwise, a time-consuming process of design, analysis, and error detection and correction has to be applied iteratively until the design of the protocol becomes error-free and consistent with the service specification. Several synthesis methods are proposed in the literature for models other than UML protocol state machines, and therefore, because of the unique features of the protocol state machines, these methods are inapplicable to services modeled in UML protocol state machines. In this paper, we propose a synthesis method that automatically synthesizes the protocol specification of distributed protocol entities from the service specification, given that both types of specifications are modeled in UML protocol state machines. Our method is based on the latest UML version (UML2.3), and it is proven to synthesize protocol specifications that are syntactically and semantically correct. As an example application, the synthesis method is used to derive the protocol specification of the H.323 standard used in Internet calls.

AUTOMATED TEST CODE GENERATION FROM CLASS STATE MODELS

This paper presents an approach to automated generation of executable test code from class models represented by the UML protocol state machines. It supports several coverage criteria for state models, including state coverage, transition coverage, and basic and extended round-trip coverage. It allows the tester to add and modify detailed test parameters (e.g., actual arguments for method invocations and implementation-specific environments) if necessary. When the state model is modified due to requirements change, the hand-crafted test parameters, if still valid, are automatically reused. This reduces the working load for regeneration of tests for modified models. In addition to test code, we also automatically generate state wrapper aspects in AspectJ, which facilitates comparing actual object states to expected states during test execution. This enables the automated verdict of pass/failure for test cases without the need to modify the source code of the class under test. We present two examples for which the executable test code is generated. They demonstrate the reuse of test parameters and testing of object interactions, respectively.

A Contract-based Approach to Specifying and Verifying Safety Critical Systems

Light-weight formal method has been regarded as an important approach to development of component-based safety critical systems. The paper proposes an approach which can formally specify and verify the contract of static structure, dynamic behavior and refinement of component systems based on UML 2.0 superstructure. As results, the correctness of static contract can be obtained via type checking of interfaces and connectors. Dynamic contract can be verified through determining the cooperativeness of integrated components, whose contracts are depicted with interface protocol state machines and their semantics models, namely contract automata. The refinement relation between high level component and its implementation will be guaranteed through defining the alternating simulation between contract automata of components at different levels.

Protocol Conformance through Refinement Mappings in Cadence SMV

This paper addresses the verification of protocol conformance between two types of state machines in model-driven software design: protocol state machines that enable specifying allowed sequences of signals and behavioral state machines that are intended for implementation specification. The contribution of the present paper, is to provide a methodology, which is based on refinement mappings, to automatically verify on protocol conformance. It helps to turn UML into a powerful and interesting tool for the development of business critical software systems.

UML-based Approach to Specify Secured, Fine-grained Concurrent Access to Shared Resources.

In object oriented paradigm, a concurrent system can be regarded as a collection of autonomous active objects which synchronize and communicate through shared passive objects. In this paper, we propose a UML-based approach to specify secured, fine-grained concurrent access to shared resources ensuring data integrity and security. The goal of the approach is to develop the UML specification with precise executional semantics, yet independent of low-level synchronization primitives and implementation environment. The approach is largely inspired from the language constructs of CDL*. A light-weight extension of UML 2.0 meta-model is proposed for the required constructs and semantics. UML protocol statemachine is used to define the access protocol for shared resources and UML activity is used to specify the behavior of methods implementing plausibly concurrent operations. The UML activity construct is extended to support concurrency features; synchronization regions, mutual exclusion and conditional synchronization not supported in current UML2.0 semantic model. The approach can be easily extended to a programming framework of design and coding.

A formal approach to model refactoring and model refinement

Model-driven engineering is an emerging software engineering approach that relies on model transformation. Typical kinds of model transformations are model refinement and model refactoring. Whenever such a transformation is applied to a consistent model, we would like to know whether the consistency is preserved by the transformation. Therefore, in this article, we formally define and explore the relation between behaviour inheritance consistency of a refined model with respect to the original model, and behaviour preservation of a refactored model with respect to the original model. As it turns out, there is a strong similarity between these notions of behaviour consistency and behaviour preservation. To illustrate this claim, we formalised the behaviour specified by UML 2.0 sequence and protocol state machine diagrams. We show how the reasoning capabilities of description logics, a decidable fragment of first-order logic, can be used in a natural way to detect behaviour inconsistencies. These reasoning capabilities can be used in exactly the same way to detect behaviour preservation violations during model refactoring. A prototype plug-in in a UML CASE tool has been developed to validate our claims.

Specifying Component Behavior with Port State Machines

Protocol State Machines (PSM) in UML 2.0 [13] describe valid sequences of operation calls. To support modeling components, UML 2.0 introduces a Port associated with a set of provided and required interfaces. Unfortunately, a PSM is applicable only to a single interface, either a provided or required one; moreover, nested calls cannot be modeled with a PSM. Furthermore, the definition of protocol conformance is rather fuzzy and reasoning on this relation is not possible in general; thus reasoning on consistency in component composition is not possible with PSMs. Behavior Protocols [17] capture the behavior of a component via a set of traces. A textual notation similar to regular expressions is provided to approximate the behavior with a regular language. In [1,17], the compliance relation and consent operator are defined to reason on consistency of component composition; a verifier tool [18] is available for the compliance relation. In this paper, we propose the Port State Machine (PoSM) to model the communication on a Port. Building on our experience with behavior protocols, we model an operation call as two atomic events request and response, permitting PoSM to capture the interleaving and nesting of operation calls on provided and required interfaces of the Port. The trace semantics of PoSM yields a regular language. We apply the compliance relation of behavior protocols to PoSMs, allowing us to reason on behavior compliance of components in software architectures; the existing verifier tool can be applied to PoSMs.

Applying Fondue to Specify a Drink Vending Machine

The purpose of the paper is to present our approach for specifying system behavior during analysis, part of the Fondue software development method. The approach is exemplified on a case study, a Drink Vending Machine (DVM). It is based on Operation Schemas and a Protocol Model. The Protocol Model describes the temporal ordering of the system operations by an UML protocol statemachine. An Operation Schema describes the functionality of a system operation by pre- and postconditions; they are written in the Object Constraint Language (OCL), with a few amendments and extensions. Our approach offers a middle ground between the informal descriptions of Use Cases and the solution-oriented models of object interaction in UML. We believe that declarative behavioral specification techniques, like the one proposed in this paper, lead to more confidence in the quality of the software because they allow one to reason about system properties.