Monitoring Program Execution Research Articles

Using memory propagation tree to improve performance of protocol fuzzer when testing ICS

Protocol fuzzers are widely used for finding vulnerabilities and security bugs in the program. The main techniques used by protocol fuzzers can be divided into 2 categories: generation-based and mutation-based fuzzing. The generation-based fuzzing generates data messages using an official specification (i.e., grammar), while the mutation-based fuzzing performs random transformations on a prepared message. But these two types of fuzzing techniques are ineffective or inefficient when testing industrial control system (ICS), because many ICS protocols are unknown, undocumented or proprietary. The generation-based fuzzing cannot work well without specifications, while the mutation-based fuzzing cannot achieve high branch coverage. In this paper, we present Miff (abbreviation of the system using “M”P tree to “i”mprove per“f”ormance of “f”uzzer) that aims at automatically abstracting data models from ICS messages. The data model generated by Miff can be used to direct protocol fuzzers to test ICS. Miff has three processing stages: (1) by instrumenting and monitoring program execution, Miff obtains the execution context information, builds memory propagation (MP) tree for every byte in the message, and identifies protocol field boundaries based on the similarity between MP trees; (2) by using information-theoretic measures, Miff infers the type of every field; (3) according to analysis results of the first two stages, Miff decides the mutation strategy for every field, which combines with the field boundary and type information to form the data model. We have implemented a prototype of Miff and applied it into 4 open-source protocol fuzzers. Our experimental results show that, Miff can enable the generation-based fuzzing to test ICS even if the specification is absent, and improve the performance of the mutation-based fuzzing to achieve higher branch coverage with less test cases.

An Abstract Interpretation-Based Model of Tracing Just-in-Time Compilation

Tracing just-in-time compilation is a popular compilation technique for the efficient implementation of dynamic languages, which is commonly used for JavaScript, Python, and PHP. It relies on two key ideas. First, it monitors program execution in order to detect so-called hot paths, that is, the most frequently executed program paths. Then, hot paths are optimized by exploiting some information on program stores that is available and therefore gathered at runtime. The result is a residual program where the optimized hot paths are guarded by sufficient conditions ensuring some form of equivalence with the original program. The residual program is persistently mutated during its execution, for example, to add new optimized hot paths or to merge existing paths. Tracing compilation is thus fundamentally different from traditional static compilation. Nevertheless, despite the practical success of tracing compilation, very little is known about its theoretical foundations. We provide a formal model of tracing compilation of programs using abstract interpretation. The monitoring phase (viz., hot path detection) corresponds to an abstraction of the trace semantics of the program that captures the most frequent occurrences of sequences of program points together with an abstraction of their corresponding stores, for example, a type environment. The optimization phase (viz., residual program generation) corresponds to a transform of the original program that preserves its trace semantics up to a given observation as modeled by some abstraction. We provide a generic framework to express dynamic optimizations along hot paths and to prove them correct. We instantiate it to prove the correctness of dynamic type specialization and constant variable folding. We show that our framework is more general than the model of tracing compilation introduced by Guo and Palsberg [2011], which is based on operational bisimulations. In our model, we can naturally express hot path reentrance and common optimizations like dead-store elimination, which are either excluded or unsound in Guo and Palsberg’s framework.

A Novel Feature Extraction Method for Monitoring (Vehicular) Fuel Storage System Leaks

System state determination with incomplete sensory information set proved to be a technically challenging problem. In this paper, authors tackle a problem of this type associated with vehicle fuel storage systems and proposed a novel feature extraction method. Federal and state regulations require fuel storage leak detection mechanism to be conducted periodically and regulate its execution rate and performance to ensure effective emission controls. Being able to robustly determine a fuel storage system’s state in terms of its effectiveness of fuel containment is therefore of great importance to all vehicle original equipment manufacturers (OEM). Prevailing practice in the industry utilizes a method relevant to natural vacuum phenomenon and is loosely associated with ideal gas law. Commonly referred to as “Entry Conditions” in in-vehicle monitoring design literature, major noise factors go through stringent pre-monitoring evaluations before monitoring program execution to ensure ideal test conditions. Differences in ambient conditions compounded with varying customer drive cycle patterns present great challenge to existing monitor designs for the purpose of leak detection. In addition, prevailing practices of evaluation in-tank fuel pressure and temperature information are generally conducted with surrogate or estmiated temperature information due to the absence of in-tank temperature sensor. All this calls for an alternative feature calculation and detection method that are less sensitive to known noise factors, can operate with incomplete sensory information yet being able provide similar or improved detection capability. In this paper, we put the main focus on the derivation of a novel method of feature calculation for the purpose of detecting presence of a leak in a fuel storage tank.

Option contracts

Many languages support behavioral software contracts so that programmers can describe a component's obligations and promises via logical assertions in its interface. The contract system monitors program execution, checks whether the assertions hold, and, if not, blames the guilty component. Pinning down the violator gets the debugging process started in the right direction. Quality contracts impose a serious run-time cost, however, and programmers therefore compromise in many ways. Some turn off contracts for deployment, but then contracts and code quickly get out of sync during maintenance. Others test contracts randomly or probabilistically. In all cases, programmers have to cope with lack of blame information when the program eventually fails. In response, we propose option contracts as an addition to the contract tool box. Our key insight is that in ordinary contract systems, server components impose their contract on client components, giving them no choice whether to trust the server's promises or check them. With option contracts, server components may choose to tag a contract as an option and clients may choose to exercise the option or accept it, in which case they also shoulder some responsibility. We show that option contracts permit programmers to specify flexible checking policies, that their cost is reasonable, and that they satisfy a complete monitoring theorem.

ConMem

Multicore technology is making concurrent programs increasingly pervasive. Unfortunately, it is difficult to deliver reliable concurrent programs, because of the huge and nondeterministic interleaving space. In reality, without the resources to thoroughly check the interleaving space, critical concurrency bugs can slip into production versions and cause failures in the field. Approaches to making the best use of the limited resources and exposing severe concurrency bugs before software release would be desirable. Unlike previous work that focuses on bugs caused by specific interleavings (e.g., races and atomicity violations), this article targets concurrency bugs that result in one type of severe effect: program crashes. Our study of the error-propagation process of real-world concurrency bugs reveals a common pattern (50% in our nondeadlock concurrency bug set) that is highly correlated with program crashes. We call this pattern concurrency-memory bugs: buggy interleavings directly cause memory bugs (NULL-pointer-dereferences, dangling-pointers, buffer-overflows, uninitialized-reads) on shared memory objects. Guided by this study, we built ConMem to monitor program execution, analyze memory accesses and synchronizations, and predictively detect these common and severe concurrency-memory bugs. We also built a validator,ConMem-v, to automatically prune false positives by enforcing potential bug-triggering interleavings. We evaluated ConMem using 7 open-source programs with 10 real-world concurrency bugs. ConMem detects more tested bugs (9 out of 10 bugs) than a lock-set-based race detector and an unserializable-interleaving detector, which detect 4 and 6 bugs, respectively, with a false-positive rate about one tenth of the compared tools. ConMem-v further prunes out all the false positives. ConMem has reasonable overhead suitable for development usage.

CDIO as a Foundation for Program Accreditation/Certification in Portugal

This document describes two initiatives for accreditation/certification of first and second cycle Bologna programs in Portugal. One initiative was started by the National Agency for Program Evaluation and Accreditation and is mandatory for all Bologna programs. The other initiative was launched by the Portuguese Professional Engineering Association and aims to certify, at the European level, Bologna second cycle engineering programs with the EUR-ACE quality seal. Both initiatives are essentially evidence-based and stress the importance of having an operating quality assurance system to support and monitor program execution. The Instituto Superior de Engenharia do Porto (ISEP) experience with the CDIO and EUR-ACE frameworks is described and strong points highlighted in this article. In the end, the authors propose that combining CDIO and EUR-ACE may bring added value, because CDIO is more oriented to program operation and EUR-ACE more oriented to program management. The authors also propose the CDIO Syllabus as the link between CDIO and EUR-ACE.

On the use of embedded debug features for permanent and transient fault resilience in microprocessors

Microprocessor-based systems are employed in an increasing number of applications where dependability is a major constraint. For this reason detecting faults arising during normal operation while introducing the least possible penalties is a main concern. Different forms of redundancy have been employed to ensure error-free behavior, while error detection mechanisms can be employed where some detection latency is tolerated. However, the high complexity and the low observability of microprocessors’ internal resources make the identification of adequate on-line error detection strategies a very challenging task, which can be tackled at circuit or system level. Concerning system-level strategies, a common limitation is in the mechanism used to monitor program execution and then detect errors as soon as possible, so as to reduce their impact on the application. In this work, an on-line error detection approach based on the reuse of available debugging infrastructures is proposed. The approach can be applied to different system architectures profiting from the debug trace port available in most of current microprocessors to observe possible misbehaviors. Two microprocessors have been used to study the applicability of the solution, LEON3 and ARM7TDMI. Results show that the presented fault detection technique enhances observability and thus error detection abilities in microprocessor-based systems without requiring modifications on the core architecture.

ConMem

Multicore technology is making concurrent programs increasingly pervasive. Unfortunately, it is difficult to deliver reliable concurrent programs, because of the huge and non-deterministic interleaving space. In reality, without the resources to thoroughly check the interleaving space, critical concurrency bugs can slip into production runs and cause failures in the field. Approaches to making the best use of the limited resources and exposing severe concurrency bugs before software release would be desirable. Unlike previous work that focuses on bugs caused by specific interleavings (e.g., races and atomicity-violations), this paper targets concurrency bugs that result in one type of severe effects: program crashes. Our study of the error-propagation process of realworld concurrency bugs reveals a common pattern (50% in our non-deadlock concurrency bug set) that is highly correlated with program crashes. We call this pattern concurrency-memory bugs: buggy interleavings directly cause memory bugs (NULL-pointer-dereference, dangling-pointer, buffer-overflow, uninitialized-read) on shared memory objects. Guided by this study, we built ConMem to monitor program execution, analyze memory accesses and synchronizations, and predicatively detect these common and severe concurrency-memory bugs. We also built a validator ConMem-v to automatically prune false positives by enforcing potential bug-triggering interleavings. We evaluated ConMem using 7 open-source programs with 9 real-world severe concurrency bugs. ConMem detects more tested bugs (8 out of 9 bugs) than a lock-set-based race detector and an unserializable-interleaving detector that detect 4 and 5 bugs respectively, with a false positive rate about one tenth of the compared tools. ConMem-v further prunes out all the false positives. ConMem has reasonable overhead suitable for development usage.

ConMem

Multicore technology is making concurrent programs increasingly pervasive. Unfortunately, it is difficult to deliver reliable concurrent programs, because of the huge and non-deterministic interleaving space. In reality, without the resources to thoroughly check the interleaving space, critical concurrency bugs can slip into production runs and cause failures in the field. Approaches to making the best use of the limited resources and exposing severe concurrency bugs before software release would be desirable. Unlike previous work that focuses on bugs caused by specific interleavings (e.g., races and atomicity-violations), this paper targets concurrency bugs that result in one type of severe effects: program crashes. Our study of the error-propagation process of realworld concurrency bugs reveals a common pattern (50% in our non-deadlock concurrency bug set) that is highly correlated with program crashes. We call this pattern concurrency-memory bugs: buggy interleavings directly cause memory bugs (NULL-pointer-dereference, dangling-pointer, buffer-overflow, uninitialized-read) on shared memory objects. Guided by this study, we built ConMem to monitor program execution, analyze memory accesses and synchronizations, and predicatively detect these common and severe concurrency-memory bugs. We also built a validator ConMem-v to automatically prune false positives by enforcing potential bug-triggering interleavings. We evaluated ConMem using 7 open-source programs with 9 real-world severe concurrency bugs. ConMem detects more tested bugs (8 out of 9 bugs) than a lock-set-based race detector and an unserializable-interleaving detector that detect 4 and 5 bugs respectively, with a false positive rate about one tenth of the compared tools. ConMem-v further prunes out all the false positives. ConMem has reasonable overhead suitable for development usage.

ECMon

The advent of multicores has introduced new challenges for programmers to provide increased performance and software reliability. There has been significant interest in techniques that use software speculation to better utilize the computational power of multicores. At the same time, several recent proposals for ensuring software reliability are not applicable in a multicore setting due to their inability to handle interprocessor shared memory dependences (ISMDs). The demands for performing speculation and ensuring software reliability in a multicore setting, although seemingly different, share a common requirement: the need for monitoring program execution and collecting interprocessor dependence information at low overhead. For example, an important component of speculation is the effcient detection of missspeculation which in turn requires dependence information. Likewise, tasks that help ensure software reliability on multicores, including recording for replay , require ISMD information. In this paper, we propose ECMon: support for exposing cache events to the software. This enables the programmer to catch these events and react to them; in effect, efficiently exposing the ISMDs to the programmer. In the context of speculation, we show how ECMon optimizes the detection of miss-speculation; we use this simple support to speculate past active barriers and achieve a speedup of 12% for the set of parallel programs considered. As an application of ensuring software reliability, we show how ECMon can be used to record shared memory dependences on multicores using no specialized hardware support at only 2.8 fold execution time overhead.

Testing and Debugging Message Passing Programs in Synergy with their Specifications

We outline Ensemble, a design and implementation methodology for composing message passing (MP) applications from program components directed by scripts. We define specification components corresponding to program components and we compose them, directed by the same scripts, obtaining formal specifications of the composed applications. We use the Petri net formalism to express component and application specifications. Petri Net composition is modelled by appropriate coupling of inscriptions on interface input and output arcs to and from a unified environment place. We elaborate on testing and debugging of MP applications based on the synergy of tools for Petri-Net simulations with tools for monitoring program executions.

An empirical comparison of monitoring algorithms for access anomaly detection

One of the major disadvantages of parallel programming with shared memory is the nondeterministic behavior caused by uncoordinated access to shared variables, known as access anomalies . Monitoring program execution to detect access anomalies is a promising and relatively unexplored approach to this problem. We present a new algorithm, referred to as task recycling , for detecting anomalies, and compare it to an existing algorithm. Empirical results indicate several significant conclusions: (i) While space requirements are bounded by Ο( T × V ), where T is the maximum number of threads that may potentially execute in parallel and V is the number of variable monitored, for typical programs space requirements are on average Ο( V ). (ii) Task recycling is more efficient in terms of space requirements and often in performance. (iii) The general approach of monitoring to detect access anomalies is practical.

An interactive high-level debugger for control-flow optimized programs

The transformations performed by an optimizing compiler have traditionally impeded interactive debugging in source language terms. A prototype system called Navigator has been developed for debugging optimized programs written in Cedar, an Algol-like language. Navigator can be used to monitor program execution flow in the presence of two optimizations: inline procedure expansion and cross-jumping (merging identical tails of code paths that join). This paper describes the problems that these two optimizations create for debugging and Navigator's solutions to these problems. The selected approach collects extra information during the optimization phases of compilation. At runtime, Navigator uses the additional information to hide the effects of the optimizations from the programmer.

An interactive high-level debugger for control-flow optimized programs

The transformations performed by an optimizing compiler have traditionally impeded interactive debugging in source language terms. A prototype system called Navigator has been developed for debugging optimized programs written in Cedar, an Algol-like language. Navigator can be used to monitor program execution flow in the presence of two optimizations: inline procedure expansion and cross-jumping (merging identical tails of code paths that join). This paper describes the problems that these two optimizations create for debugging and Navigator's solutions to these problems. The selected approach collects extra information during the optimization phases of compilation. At runtime, Navigator uses the additional information to hide the effects of the optimizations from the programmer.

Special Feature: Monitoring Program Execution: A Survey

Progress in this realm of performance measurement has lagged behind advances in the general field of programming languages: careful study of its capabilities and limitations is long overdue.

A Flexible and Inexpensive Method of Monitoring Program Execution in a Digital Computer

A method of monitoring the program execution in a digital computer on the basis of the flow diagram of the computing program has been devised. A comparatively low-cost equipment for monitoring a maximum of 64 boxes in a flow diagram has been constructed. The monitoring method is flexible and convenient in its application. It can be used in connection with relative or symbolic addresses, compilers, etc. The user must provide only a flow diagram drawn on translucent paper in a certain form and the information to correlate this diagram with the computing program. A subroutine modifies the computing program as needed for the monitoring purpose and restores it to its original form when the user so desires. The monitoring introduces only a very small increase in computing time, requiring for each call-up of a box in the flow diagram only a time amounting to that needed for two simple unconditional jumps. The monitor can be used to present during the computation a visual dynamic picture of the progress of the program and to register, on occurrence, the whereabouts of an interruption, thus facilitating the tracing of the error. The principle of the monitoring method and the subroutine program, and the essentials of the constructed monitor equipment, are described in detail.