Program Synthesis Research Articles

Program Boosting

In this paper, we investigate an approach to program synthesis that is based on crowd-sourcing. With the help of crowd-sourcing, we aim to capture the "wisdom of the crowds" to find good if not perfect solutions to inherently tricky programming tasks, which elude even expert developers and lack an easy-to-formalize specification. We propose an approach we call program boosting, which involves crowd-sourcing imperfect solutions to a difficult programming problem from developers and then blending these programs together in a way that improves their correctness. We implement this approach in a system called CROWDBOOST and show in our experiments that interesting and highly non-trivial tasks such as writing regular expressions for URLs or email addresses can be effectively crowd-sourced. We demonstrate that carefully blending the crowd-sourced results together consistently produces a boost, yielding results that are better than any of the starting programs. Our experiments on 465 program pairs show consistent boosts in accuracy and demonstrate that program boosting can be performed at a relatively modest monetary cost.

Toward synthesizing executable models in biology.

Over the last decade, executable models of biological behaviors have repeatedly provided new scientific discoveries, uncovered novel insights, and directed new experimental avenues. These models are computer programs whose execution mechanistically simulates aspects of the cell’s behaviors. If the observed behavior of the program agrees with the observed biological behavior, then the program explains the phenomena. This approach has proven beneficial for gaining new biological insights and directing new experimental avenues. One advantage of this approach is that techniques for analysis of computer programs can be applied to the analysis of executable models. For example, one can confirm that a model agrees with experiments for all possible executions of the model (corresponding to all environmental conditions), even if there are a huge number of executions. Various formal methods have been adapted for this context, for example, model checking or symbolic analysis of state spaces. To avoid manual construction of executable models, one can apply synthesis, a method to produce programs automatically from high-level specifications. In the context of biological modeling, synthesis would correspond to extracting executable models from experimental data. We survey recent results about the usage of the techniques underlying synthesis of computer programs for the inference of biological models from experimental data. We describe synthesis of biological models from curated mutation experiment data, inferring network connectivity models from phosphoproteomic data, and synthesis of Boolean networks from gene expression data. While much work has been done on automated analysis of similar datasets using machine learning and artificial intelligence, using synthesis techniques provides new opportunities such as efficient computation of disambiguating experiments, as well as the ability to produce different kinds of models automatically from biological data.

A practical comparator of cost functions and its applications

Automatic cost analysis has significantly advanced in the last few years. Nowadays, a number of cost analyzers exist which automatically produce upper- and/or lower-bounds on the amount of resources required to execute a program. Cost analysis has a number of important applications such as resource-usage verification and program synthesis and optimization. For such applications to be successful, it is not sufficient to have automatic cost analysis. It is also required to have automated means for handling the analysis results, which are in the form of Cost Functions (CFs for short) i.e., non-recursive expressions composed of a relatively small number of types of basic expressions. In particular, we need automated means for comparing CFs in order to prove that a CF is smaller than or equal to another one for all input values of interest. General function comparison is a hard mathematical problem. Rather than attacking the general problem, in this work we focus on comparing CFs by exploiting their syntactic properties and we present, to the best of our knowledge, the first practical CF comparator which opens the door to fully automated applications of cost analysis. We have implemented the comparator and made its source code available online, so that any cost analyzer can use it.

Synthesizing bounded-time 2-phase fault recovery

Abstract We focus on synthesis techniques for transforming existing fault-intolerant real-time programs into fault-tolerant programs that provide phased recovery . A fault-tolerant program is one that satisfies its safety and liveness specifications as well as timing constraints in the presence of faults. We argue that in many commonly considered programs (especially in safety/mission-critical systems), when faults occur, simple recovery to the program’s normal behavior is necessary, but not sufficient. For such programs, it is necessary that recovery is accomplished in a sequence of phases, each ensuring that the program satisfies certain properties. In the simplest case, in the first phase the program recovers to an acceptable behavior within some time θ , and, in the second phase, it recovers to the ideal behavior within time δ . In this article, we introduce four different types of bounded-time 2-phase recovery, namely ordered-strict, strict, relaxed, and graceful, based on how a real-time fault-tolerant program reaches the acceptable and ideal behaviors in the presence of faults. We rigorously analyze the complexity of automated synthesis of each type: we either show that the problem is hard in some class of complexity or we present a sound and complete synthesis algorithm. We argue that such complexity analysis is essential to deal with the highly complex decision procedures of program synthesis.

Program Synthesis for Network Updates

Program Synthesis for Network Updates

Smten with satisfiability-based search

Satisfiability (SAT) and Satisfiability Modulo Theories (SMT) have been used in solving a wide variety of important and challenging problems, including automatic test generation, model checking, and program synthesis. For these applications to scale to larger problem instances, developers cannot rely solely on the sophistication of SAT and SMT solvers to efficiently solve their queries; they must also optimize their own orchestration and construction of queries. We present Smten, a high-level language for orchestrating and constructing satisfiability-based search queries. We show that applications developed using Smten require significantly fewer lines of code and less developer effort to achieve results comparable to standard SMT-based tools.

Ozone vegetation damage effects on gross primary productivity in the United States

Abstract. We apply an off-line process-based vegetation model (the Yale Interactive Terrestrial Biosphere model) to assess the impacts of ozone (O3) vegetation damage on gross primary productivity (GPP) in the United States during the past decade (1998–2007). The model's GPP simulation is evaluated at 40 sites of the North American Carbon Program (NACP) synthesis. The ecosystem-scale model version reproduces interannual variability and seasonality of GPP at most sites, especially in croplands. Inclusion of the O3 damage impact decreases biases of simulated GPP at most of the NACP sites. The simulation with the O3 damage effect reproduces 64% of the observed variance in summer GPP and 42% on the annual average. Based on a regional gridded simulation over the US, summertime average O3-free GPP is 6.1 g C m−2 day−1 (9.5 g C m−2 day−1 in the east of 95° W and 3.9 g C m−2 day−1 in the west). O3 damage decreases GPP by 4–8% on average in the eastern US and leads to significant decreases of 11–17% in east coast hot spots. Sensitivity simulations show that a 25% decrease in surface O3 concentration halves the average GPP damage to only 2–4%, suggesting the substantial co-benefits to ecosystem health that may be achieved via O3 air pollution control.

Using formal methods to enable more secure vehicles

Networked embedded systems are ubiquitous in modern society. Examples include SCADA systems that manage physical infrastructure, medical devices such as pacemakers and insulin pumps, and vehicles such as airplanes and automobiles. Such devices are connected to networks for a variety of compelling reasons, including the ability to access diagnostic information conveniently, perform software updates, provide innovative features, and lower costs. Researchers and hackers have shown that these kinds of networked embedded systems are vulnerable to remote attacks and that such attacks can cause physical damage and can be hidden from monitors [1, 4]. DARPA launched the HACMS program to create technology to make such systems dramatically harder to attack successfully. Specifically, HACMS is pursuing a clean-slate, formal methods-based approach to the creation of high-assurance vehicles, where high assurance is defined to mean functionally correct and satisfying appropriate safety and security properties. Specific technologies include program synthesis, domain-specific languages, and theorem provers used as program development environments. Targeted software includes operating system components such as hypervisors, microkernels, file systems, and device drivers as well as control systems such as autopilots and adaptive cruise controls. Program researchers are leveraging existing high-assurance software including NICTA's seL4 microkernel and INRIA's CompCert compiler. Although the HACMS project is less than halfway done, the program has already achieved some remarkable success. At program kick-off, a Red Team easily hijacked the baseline open-source quadcopter that HACMS researchers are using as a research platform. At the end of eighteen months, the Red Team was not able to hijack the newly-minted "SMACCMCopter" running high-assurance HACMS code, despite being given six weeks and full access to the source code of the copter. An expert in penetration testing called the SMACCMCopter "the most secure UAV on the planet." In this talk, I will describe the HACMS program: its motivation, the underlying technologies, current results, and future directions.

Deductive Synthesis of Algorithms in Human-Computer Interaction

Deductive synthesis is a method of software development where an algorithm is derived from a formal problem specification which guarantees the reliability of final product. The paper introduces a program synthesis method PAR most of whose synthesis steps are mechanical and some of them can be done interactively by human-computer interaction, and formally synthesizes a dependable algorithm for a selection problem supported by PAR method and PAR platform. Program synthesis based on PAR covers a number of classical algorithm design tactics, develops algorithmic programs together with their proof of correctness, and makes the algorithm more reliable and solving idea more understandable.

Program Synthesis and Linear Operator Semantics

For deterministic and probabilistic programs we investigate the problem of program synthesis and program optimisation (with respect to non-functional properties) in the general setting of global optimisation. This approach is based on the representation of the semantics of programs and program fragments in terms of linear operators, i.e. as matrices. We exploit in particular the fact that we can automatically generate the representation of the semantics of elementary blocks. These can then can be used in order to compositionally assemble the semantics of a whole program, i.e. the generator of the corresponding Discrete Time Markov Chain (DTMC). We also utilise a generalised version of Abstract Interpretation suitable for this linear algebraic or functional analytical framework in order to formulate semantical constraints (invariants) and optimisation objectives (for example performance requirements).

An integrated programming and development environment for adiabatic quantum optimization

Adiabatic quantum computing is a promising route to the computational power afforded by quantum information processing. The recent availability of adiabatic hardware has raised challenging questions about how to evaluate adiabatic quantum optimization programs. Processor behavior depends on multiple steps to synthesize an adiabatic quantum program, which are each highly tunable. We present an integrated programming and development environment for adiabatic quantum optimization called JADE that provides control over all the steps taken during program synthesis. JADE captures the workflow needed to rigorously specify the adiabatic quantum optimization algorithm while allowing a variety of problem types, programming techniques, and processor configurations. We have also integrated JADE with a quantum simulation engine that enables program profiling using numerical calculation. The computational engine supports plug-ins for simulation methodologies tailored to various metrics and computing resources. We present the design, integration, and deployment of JADE and discuss its potential use for benchmarking adiabatic quantum optimization programs by the quantum computer science community.

From Requirements to Ready to Run Software: A Brief Thought on How to Mechanize The Software Development Process

This paper presents a vision on how the software development process could be a fully unified mechanized process by getting benefits from the advances of Natural Language Processing and Program Synthesis fields. The process begins from requirements written in natural language that is translated to sentences in logical form. A program synthesizer gets those sentences in logical form (the translator's outcome) and generates a source code. Finally, a compiler produces ready-to-run software. To find out how the building blocks of our proposed approach works, we conducted an exploratory research on the literature in the fields of Requirements Engineering, Natural Language Processing, and Program Synthesis. Currently, this approach is difficult to accomplish in a fully automatic way due to the ambiguities inherent in natural language, the reasoning context of the software, and the program synthesizer limitations in generating a source code from logic.

A modular order-sorted equational generalization algorithm

Generalization, also called anti-unification, is the dual of unification. Given terms t and t′, a generalizer is a term t″ of which t and t′ are substitution instances. The dual of a most general unifier (mgu) is that of least general generalizer (lgg). In this work, we extend the known untyped generalization algorithm to, first, an order-sorted typed setting with sorts, subsorts, and subtype polymorphism; second, we extend it to work modulo equational theories, where function symbols can obey any combination of associativity, commutativity, and identity axioms (including the empty set of such axioms); and third, to the combination of both, which results in a modular, order-sorted equational generalization algorithm. Unlike the untyped case, there is in general no single lgg in our framework, due to order-sortedness or to the equational axioms. Instead, there is a finite, minimal and complete set of lggs, so that any other generalizer has at least one of them as an instance. Our generalization algorithms are expressed by means of inference systems for which we give proofs of correctness. This opens up new applications to partial evaluation, program synthesis, and theorem proving for typed equational reasoning systems and typed rule-based languages such as ASF+SDF, Elan, OBJ, Cafe-OBJ, and Maude.

Symbolic optimization with SMT solvers

The rise in efficiency of Satisfiability Modulo Theories (SMT) solvers has created numerous uses for them in software verification, program synthesis, functional programming, refinement types, etc. In all of these applications, SMT solvers are used for generating satisfying assignments (e.g., a witness for a bug) or proving unsatisfiability/validity(e.g., proving that a subtyping relation holds). We are often interested in finding not just an arbitrary satisfying assignment, but one that optimizes (minimizes/maximizes) certain criteria. For example, we might be interested in detecting program executions that maximize energy usage (performance bugs), or synthesizing short programs that do not make expensive API calls. Unfortunately, none of the available SMT solvers offer such optimization capabilities. In this paper, we present SYMBA, an efficient SMT-based optimization algorithm for objective functions in the theory of linear real arithmetic (LRA). Given a formula φ and an objective function t , SYMBA finds a satisfying assignment of φthat maximizes the value of t . SYMBA utilizes efficient SMT solvers as black boxes. As a result, it is easy to implement and it directly benefits from future advances in SMT solvers. Moreover, SYMBA can optimize a set of objective functions, reusing information between them to speed up the analysis. We have implemented SYMBA and evaluated it on a large number of optimization benchmarks drawn from program analysis tasks. Our results indicate the power and efficiency of SYMBA in comparison with competing approaches, and highlight the importance of its multi-objective-function feature.

A constraint-based approach to solving games on infinite graphs

We present a constraint-based approach to computing winning strategies in two-player graph games over the state space of infinite-state programs. Such games have numerous applications in program verification and synthesis, including the synthesis of infinite-state reactive programs and branching-time verification of infinite-state programs. Our method handles games with winning conditions given by safety, reachability, and general Linear Temporal Logic (LTL) properties. For each property class, we give a deductive proof rule that --- provided a symbolic representation of the game players --- describes a winning strategy for a particular player. Our rules are sound and relatively complete. We show that these rules can be automated by using an off-the-shelf Horn constraint solver that supports existential quantification in clause heads. The practical promise of the rules is demonstrated through several case studies, including a challenging "Cinderella-Stepmother game" that allows infinite alternation of discrete and continuous choices by two players, as well as examples derived from prior work on program repair and synthesis.

Intelligent Computing on the Basis of Cognitive and Event Modeling, and Its Application in Energy Security Research

The paper considers the issues of implementation and application of intelligent computing on the basis of cognitive and event modeling in research on energy security. The authors suggest a two-level information technology for the research. The first level suggests a situation analysis using the intelligent computing techniques. The analysis results are then used to choose rational variants of energy development in Russia (or its regions). At the second level these variants are computed with the multi-agent software INTEC-M. Transition from the first to the second level is automated by the tools of deductive program synthesis, that are based on declarative descriptions, i.e. formulae of restricted predicate calculus, and representation of input data by XML files. Cognitive and event modeling is considered in more detail. The examples of cognitive and event models are presented. The structure of a knowledge space is developed to support the intelligent computations. The knowledge space includes ontological models, databases of cognitive and event models, and the database on the cases of energy emergency situations. The authors developed the CogMap and EventMap tools to support cognitive and event modeling on the basis of common graphical environment GirModeling, and the expert system “Emergency”. The tools and expert system that support the knowledge base on energy emergencies are integrated within the intelligent IT environment. The research presented in the paper was partially supported by the grant of Presidium of RAS No. 2.2-2012 and grants of Russian Foundation of Basic Research No. 10-07-00264, No. 11-07-00192, No. 11-07-00245, and No. 12-07-00359.

Efficient program synthesis using constraint satisfaction in inductive logic programming

We present NrSample, a framework for program synthesis in inductive logic programming. NrSample uses propositional logic constraints to exclude undesirable candidates from the search. This is achie...

Конструктивный синтез пользовательских интерфейсов Web-приложений

The article discusses the application of the method of constructive 
 program synthesis to create user interfaces for Web-based applications. Specialized 
 constructive logic is introduced, which allows to synthesize models of guaranteed 
 right user interfaces. We consider the software implementation and results of the 
 practical application of the model.

Engineering domain-specific languages with formula 2.0

Domain-specific languages (DSLs) are useful for capturing and reusing engineering expertise. They can formalize industrial patterns and practices while increasing the scalability of verification, because input programs are written at a higher level of abstraction. However, engineering new DSLs with custom verification is a non-trivial task in its own right, and usually requires programming language, formal methods, and automated theorem proving expertise. In this tutorial we present FORMULA 2.0, which is formal framework for developing DSLs. FORMULA specifications are succinct descriptions of DSLs, and specifications can be immediately connected to state-of-the-art analysis engines without additional expertise. FORMULA provides: (1) succinct specifications of DSLs and compilers, (2) efficient compilation and execution of input programs, (3) program synthesis and compiler verification. We take a unique approach to provide these features: Specifications are written as strongly-typed open-world logic programs. These specifications are highly declarative and easily express rich synthesis / verification problems. Automated reasoning is enabled by efficient symbolic execution of logic programs into quantifier-free sub-problems, which are dispatched to the state-of-the-art SMT solver Z3. FORMULA has been applied within Microsoft to develop DSLs for verifiable device drivers and protocols. It has been used by the automotive / embedded systems industries for software / hardware co-design and design-space exploration under hard resource allocation constraints. It is being used to develop semantic specifications for complex cyber-physical systems.

A realist synthesis of the effect of social accountability interventions on health service providers’ and policymakers’ responsiveness

BackgroundAccountability has center stage in the current post-Millennium Development Goals (MDG) debate. One of the effective strategies for building equitable health systems and providing quality health services is the strengthening of citizen-driven or social accountability processes. The monitoring of actions and decisions of policymakers and providers by citizens is regarded as a right in itself but also as an alternative to weak administrative accountability mechanisms, in particular in settings with poor governance. The effects of social accountability interventions are often based on assumptions and are difficult to evaluate because of their complex nature and context sensitivity. This study aims to review and assess the available evidence for the effect of social accountability interventions on policymakers’ and providers’ responsiveness in countries with medium to low levels of governance capacity and quality. For policymakers and practitioners engaged in health system strengthening, social accountability initiatives and rights-based approaches to health, the findings of this review may help when reflecting on the assumptions and theories of change behind their policies and interventions.Methods/DesignLittle is known about social accountability interventions, their outcomes and the circumstances under which they produce outcomes for particular groups or issues. In this study, social accountability interventions are conceptualized as complex social interventions for which a realist synthesis is considered the most appropriate method of systematic review. The synthesis is based on a preliminary program theory of social accountability that will be tested through an iterative process of primary study searches, data extraction, analysis and synthesis. Published and non-published (grey) quantitative and qualitative studies in English, French and Spanish will be included. Quality and validity will be enhanced by continuous peer review and team reflection among the reviewers.DiscussionThe authors believe the advantages of a realist synthesis for social accountability lie in the possibility of overcoming disciplinary or paradigmatic boundaries often found in public health and development. In addition, they argue that this approach fills the knowledge gap left by conventional synthesis or evaluation exercises of participatory programs. Finally, the authors describe the practical strategies adopted to address methodological challenges and validity.