First-order Programs Research Articles

Error Credits: Resourceful Reasoning about Error Bounds for Higher-Order Probabilistic Programs

Probabilistic programs often trade accuracy for efficiency, and thus may, with a small probability, return an incorrect result. It is important to obtain precise bounds for the probability of these errors, but existing verification approaches have limitations that lead to error probability bounds that are excessively coarse, or only apply to first-order programs. In this paper we present Eris, a higher-order separation logic for proving error probability bounds for probabilistic programs written in an expressive higher-order language. Our key novelty is the introduction of error credits, a separation logic resource that tracks an upper bound on the probability that a program returns an erroneous result. By representing error bounds as a resource, we recover the benefits of separation logic, including compositionality, modularity, and dependency between errors and program terms, allowing for more precise specifications. Moreover, we enable novel reasoning principles such as expectation-preserving error composition, amortized error reasoning, and error induction. We illustrate the advantages of our approach by proving amortized error bounds on a range of examples, including collision probabilities in hash functions, which allow us to write more modular specifications for data structures that use them as clients. We also use our logic to prove correctness and almost-sure termination of rejection sampling algorithms. All of our results have been mechanized in the Coq proof assistant using the Iris separation logic framework and the Coquelicot real analysis library.

Distributions for Compositionally Differentiating Parametric Discontinuities

Computations in physical simulation, computer graphics, and probabilistic inference often require the differentiation of discontinuous processes due to contact, occlusion, and changes at a point in time. Popular differentiable programming languages, such as PyTorch and JAX, ignore discontinuities during differentiation. This is incorrect for parametric discontinuities —conditionals containing at least one real-valued parameter and at least one variable of integration. We introduce Potto, the first differentiable first-order programming language to soundly differentiate parametric discontinuities. We present a denotational semantics for programs and program derivatives and show the two accord. We describe the implementation of Potto, which enables separate compilation of programs. Our prototype implementation overcomes previous compile-time bottlenecks achieving an 88.1x and 441.2x speed up in compile time and a 2.5x and 7.9x speed up in runtime, respectively, on two increasingly large image stylization benchmarks. We showcase Potto by implementing a prototype differentiable renderer with separately compiled shaders.

Integrated Pest Management Protocols for Space-Based Bioregenerative Life Support Systems

Human missions to the Moon and Mars will necessarily increase in both duration and complexity over the coming decades. In the past, short-term missions to low-Earth orbit (LEO) or the Moon (e.g., Apollo) utilized physiochemical life support systems for the crews. However, as the spatial and temporal durations of crewed missions to other planetary bodies increase, physiochemical life support systems become burdened with the requirement of frequent resupply missions. Bioregenerative life support systems (BLSS) have been proposed to replace much of the resupply required of physiochemical systems with modules that can regenerate water, oxygen, and food stocks with plant-based biological production systems. In order to protect the stability and productivity of BLSS modules (i.e., small scale units) or habitats (i.e., large scale systems), an integrated pest management (IPM) program is required to prevent, mitigate, and eliminate both insect pests and disease outbreaks in space-based plant-growing systems. A first-order BLSS IPM program is outlined herein that summarizes a collection of protocols that are similar to those used in field, greenhouse, and vertical-farming agricultural systems. However, the space environment offers numerous unusual stresses to plants, and thus, unique space-based IPM protocols will have to be developed. In general, successful operation of space-based BLSS units will be guided by IPM protocols that (1) should be established early in the mission design phase to be effective, (2) will be dynamic in nature changing both spatially and temporally depending on the successional processes afoot within the crewed spacecraft, plant-growing systems, and through time; and (3) can prevent insect/phytopathology outbreaks at very high levels that can approach 100% if properly implemented.

A convex programming solution for gate-sizing with pipelining constraints

This paper proposes a novel geometric programming based formulation to solve a gate-sizing and retiming problem in the context of circuit optimization. The gate-sizing aspect of the problem involves continuous variables while the retiming problem involves the placement of registers in the circuit and can be naturally modeled using discrete variables. Our formulation is solved using first-order convex programming. We show promising experimental results on industrial circuits. We also investigate formally the computational complexity of the problem. To our knowledge, this is the first effort that solves this problem in a single optimization framework.

Learning Higher-Order Programs through Predicate Invention

A key feature of inductive logic programming (ILP) is its ability to learn first-order programs, which are intrinsically more expressive than propositional programs. In this paper, we introduce ILP techniques to learn higher-order programs. We implement our idea in Metagolho, an ILP system which can learn higher-order programs with higher-order predicate invention. Our experiments show that, compared to first-order programs, learning higher-order programs can significantly improve predictive accuracies and reduce learning times.

Backpropagation in the simply typed lambda-calculus with linear negation

Backpropagation is a classic automatic differentiation algorithm computing the gradient of functions specified by a certain class of simple, first-order programs, called computational graphs. It is a fundamental tool in several fields, most notably machine learning, where it is the key for efficiently training (deep) neural networks. Recent years have witnessed the quick growth of a research field called differentiable programming, the aim of which is to express computational graphs more synthetically and modularly by resorting to actual programming languages endowed with control flow operators and higher-order combinators, such as map and fold. In this paper, we extend the backpropagation algorithm to a paradigmatic example of such a programming language: we define a compositional program transformation from the simply-typed lambda-calculus to itself augmented with a notion of linear negation, and prove that this computes the gradient of the source program with the same efficiency as first-order backpropagation. The transformation is completely effect-free and thus provides a purely logical understanding of the dynamics of backpropagation.

Learning higher-order logic programs

A key feature of inductive logic programming is its ability to learn first-order programs, which are intrinsically more expressive than propositional programs. In this paper, we introduce techniques to learn higher-order programs. Specifically, we extend meta-interpretive learning (MIL) to support learning higher-order programs by allowing for higher-order definitions to be used as background knowledge. Our theoretical results show that learning higher-order programs, rather than first-order programs, can reduce the textual complexity required to express programs, which in turn reduces the size of the hypothesis space and sample complexity. We implement our idea in two new MIL systems: the Prolog system text {Metagol}_{ho} and the ASP system text {HEXMIL}_{ho}. Both systems support learning higher-order programs and higher-order predicate invention, such as inventing functions for map/3 and conditions for filter/3. We conduct experiments on four domains (robot strategies, chess playing, list transformations, and string decryption) that compare learning first-order and higher-order programs. Our experimental results support our theoretical claims and show that, compared to learning first-order programs, learning higher-order programs can significantly improve predictive accuracies and reduce learning times.

Higher-order Demand-driven Program Analysis

Developing accurate and efficient program analyses for languages with higher-order functions is known to be difficult. Here we define a new higher-order program analysis, Demand-Driven Program Analysis (DDPA), which extends well-known demand-driven lookup techniques found in first-order program analyses to higher-order programs. This task presents several unique challenges to obtain good accuracy, including the need for a new method for demand-driven lookup of non-local variable values. DDPA is flow- and context-sensitive and provably polynomial-time. To efficiently implement DDPA, we develop a novel pushdown automaton metaprogramming framework, the Pushdown Reachability automaton. The analysis is formalized and proved sound, and an implementation is described.

Maximum likelihood estimation of nonlinear mixed-effects models with crossed random effects by combining first-order conditional linearization and sequential quadratic programming

Nonlinear mixed-effects (NLME) models have become popular in various disciplines over the past several decades. However, the existing methods for parameter estimation implemented in standard statistical packages such as SAS and R/S-Plus are generally limited to single- or multi-level NLME models that only allow nested random effects and are unable to cope with crossed random effects within the framework of NLME modeling. In this study, we propose a general formulation of NLME models that can accommodate both nested and crossed random effects, and then develop a computational algorithm for parameter estimation based on normal assumptions. The maximum likelihood estimation is carried out using the first-order conditional expansion (FOCE) for NLME model linearization and sequential quadratic programming (SQP) for computational optimization while ensuring positive-definiteness of the estimated variance-covariance matrices of both random effects and error terms. The FOCE-SQP algorithm is evaluated using the height and diameter data measured on trees from Korean larch (L. olgensis var. Changpaiensis) experimental plots as well as simulation studies. We show that the FOCE-SQP method converges fast with high accuracy. Applications of the general formulation of NLME models are illustrated with an analysis of the Korean larch data.

Reliability Assessment of Bearing Capacity of Cement–Iron Ore Tailing Blend Black Cotton Soil for Strip Foundations

A reliability estimate of bearing capacity values of cement–iron ore tailing treated black cotton soil was evaluated using a predictive model developed from laboratory results for specimens compacted at British Standard light, West African standard and British Standard heavy energy levels. Laboratory based geotechnical properties of the treated black cotton soil were incorporated into FORTRAN-based first-order reliability program to determine the reliability index values for the following parameters: cement content, C, Iron ore tailing content, IOT, cohesion, CO, friction angle, FA, percentage fine, PF, plasticity index, PI and unit weight, UW. Although cohesion, friction angle, plasticity index and unit weight were greatly influenced by the coefficient of variation, cement content C, IOT content IOT and percentage fine PF did not significantly affect the reliability index values. F-distribution test at 95% level of significance shows that compactive effort had significant effect on the outcome of the reliability index values. However, since reliability index (β) value of 1.0, which is considered adequate for serviceability limit state design, was not met, none of the compactive efforts could be used to model the behaviour of bearing capacity of cement–IOT treated black cotton soil. Hence, a more potent admixture should be used for improving the bearing capacity of strip foundation.

Logic programming: Laxness and saturation

A propositional logic program P may be identified with a PfPf-coalgebra on the set of atomic propositions in the program. The corresponding C(PfPf)-coalgebra, where C(PfPf) is the cofree comonad on PfPf, describes derivations by resolution. That correspondence has been developed to model first-order programs in two ways, with lax semantics and saturated semantics, based on locally ordered categories and right Kan extensions respectively. We unify the two approaches, exhibiting them as complementary rather than competing, reflecting the theorem-proving and proof-search aspects of logic programming. While maintaining that unity, we further refine lax semantics to give finitary models of logic programs with existential variables, and to develop a precise semantic relationship between variables in logic programming and worlds in local state.

A Logical Analysis of Framing for Specifications with Pure Method Calls

For specifying and reasoning about object-based programs, it is often attractive for contracts to be expressed using calls to pure methods. It is useful for pure methods to have contracts, including read effects, to support local reasoning based on frame conditions. This leads to puzzles such as the use of a pure method in its own contract. These ideas have been explored in connection with verification tools based on axiomatic semantics, guided by the need to avoid logical inconsistency, and focusing on encodings that cater for first-order automated provers. This article adds pure methods and read effects to region logic, a first-order program logic that features frame-based local reasoning and provides modular reasoning principles for end-to-end correctness. Modular reasoning is embodied in a proof rule for linking a module’s method implementations with a client that relies on the method contracts. Soundness is proved with respect to conventional operational semantics and uses an extensional (i.e, relational) interpretation of read effects. Applicability to tools based on SMT solvers is demonstrated through machine-checked verification of examples. The developments in this article can guide the implementations of linking as used in modular verifiers and serve as a basis for studying observationally pure methods and encapsulation.

An Improved Interval Fuzzy Modeling Method: Applications to the Estimation of Photovoltaic/Wind/Battery Power in Renewable Energy Systems

This paper proposes an improved interval fuzzy modeling (imIFML) technique based on modified linear programming and actual boundary points of data. The imIFML technique comprises four design stages. The first stage is based on conventional interval fuzzy modeling (coIFML) with first-order model and linear programming. The second stage defines reference lower and upper bounds of data using MATLAB. The third stage initially adjusts scaling parameters in the modified linear programming. The last stage automatically fine-tunes parameters in the modified linear programming to realize the best possible model. Lower and upper bounds approximated by the imIFML technique are closely fitted to the reference lower and upper bounds, respectively. The proposed imIFML is thus significantly less conservative in cases of large variation in data, while robustness is inherited from the coIFML. Design flowcharts, equations, and sample MATLAB code are presented for reference in future experiments. Performance and efficacy of the introduced imIFML are evaluated to estimate solar photovoltaic, wind and battery power in a demonstrative renewable energy system under large data changes. The effectiveness of the proposed imIFML technique is also compared with the coIFML technique.

Effect-dependent transformations for concurrent programs

• Denotational semantics for an effect system for concurrency. • The Semantics validates a number of new effect-dependent program equivalences. • In comparison to the conference version: more detail about the higher-order version of Brookes' trace semantics (Section 3). • More examples, in particular the one on loop parallelization. • Detailed proofs of the main results, in particular soundness of the logical relation and general reasoning principles. We describe a denotational semantics for an abstract effect system for a higher-order, shared-variable concurrent programming language. We prove the soundness of a number of general effect-based program equivalences, including a parallelization equation that specifies sufficient conditions for replacing sequential composition with parallel composition. Effect annotations are relative to abstract locations specified by contracts rather than physical footprints allowing us in particular to show the soundness of some transformations involving fine-grained concurrent data structures, such as Michael–Scott queues, that allow concurrent access to different parts of mutable data structures. Our semantics is based on refining a trace-based semantics for first-order programs due to Brookes. By moving from concrete to abstract locations, and adding type refinements that capture the possible side-effects of both expressions and their concurrent environments, we are able to validate many equivalences that do not hold in an unrefined model. The meanings of types are expressed using a game-based logical relation over sets of traces. Two programs e 1 and e 2 are logically related if one is able to solve a two-player game: for any trace with result value v 1 in the semantics of e 1 (challenge) that the player presents, the opponent can present an (response) equivalent trace in the semantics of e 2 with a logically related result value v 2 .

On the Minimization Problem for Sequential Programs

First-order program schemata represent one of the most simple models of sequential imperative programs intended for solving verification and optimization problems. We consider the decidable relation of logical-thermal equivalence on these schemata and the problem of their size minimization while preserving logical-thermal equivalence. We prove that this problem is decidable. Further we show that the first-order program schemata supplied with logical-thermal equivalence and finite-state deterministic transducers operating over substitutions are mutually translated into each other. This relationship makes it possible to adapt equivalence checking and minimization algorithms developed in one of these models of computation to the solution of the same problems for the other model of computation. In addition, on the basis of the discovered relationship, we describe a subclass of first-order program schemata such that minimization of the program schemata from this class can be performed in polynomial time by means of known techniques for minimization of finite-state transducers operating over semigroups. Finally, we demonstrate that in the general case the minimization problem for finite state transducers over semigroups may have several non-isomorphic solutions.

On the Minimization Problem for Sequential Programs

First-order program schemata is one of the simplest models of sequential imperative programs intended for solving verification and optimization problems. We consider the decidable relation of logical-thermal equivalence of these schemata and the problem of their size minimization while preserving logical-thermal equivalence. We prove that this problem is decidable. Further we show that the first-order program schemata supplied with logical-thermal equivalence and finite state deterministic transducers operating over substitutions are mutually translated into each other. This relationship implies that the equivalence checking problem and the minimization problem for these transducers are also decidable. In addition, on the basis of the discovered relationship, we have found a subclass of firstorder program schemata such that their minimization can be performed in polynomial time by means of known techniques for minimization of finite state transducers operating over semigroups. Finally, we demonstrate that in general case the minimization problem for finite state transducers over semigroups may have several non-isomorphic solutions.

Contract-based resource verification for higher-order functions with memoization

We present a new approach for specifying and verifying resource utilization of higher-order functional programs that use lazy evaluation and memoization. In our approach, users can specify the desired resource bound as templates with numerical holes e.g. as steps ≤ ? * size(l) + ? in the contracts of functions. They can also express invariants necessary for establishing the bounds that may depend on the state of memoization. Our approach operates in two phases: first generating an instrumented first-order program that accurately models the higher-order control flow and the effects of memoization on resources using sets, algebraic datatypes and mutual recursion, and then verifying the contracts of the first-order program by producing verification conditions of the form ∃ ∀ using an extended assume/guarantee reasoning. We use our approach to verify precise bounds on resources such as evaluation steps and number of heap-allocated objects on 17 challenging data structures and algorithms. Our benchmarks, comprising of 5K lines of functional Scala code, include lazy mergesort , Okasaki's real-time queue and deque data structures that rely on aliasing of references to first-class functions; lazy data structures based on numerical representations such as the conqueue data structure of Scala's data-parallel library, cyclic streams, as well as dynamic programming algorithms such as knapsack and Viterbi . Our evaluations show that when averaged over all benchmarks the actual runtime resource consumption is 80% of the value inferred by our tool when estimating the number of evaluation steps, and is 88% for the number of heap-allocated objects.

Towards automatic resource bound analysis for OCaml

This article presents a resource analysis system for OCaml programs. The system automatically derives worst-case resource bounds for higher-order polymorphic programs with user-defined inductive types. The technique is parametric in the resource and can derive bounds for time, memory allocations and energy usage. The derived bounds are multivariate resource polynomials which are functions of different size parameters that depend on the standard OCaml types. Bound inference is fully automatic and reduced to a linear optimization problem that is passed to an off-the-shelf LP solver. Technically, the analysis system is based on a novel multivariate automatic amortized resource analysis (AARA). It builds on existing work on linear AARA for higher-order programs with user-defined inductive types and on multivariate AARA for first-order programs with built-in lists and binary trees. This is the first amortized analysis, that automatically derives polynomial bounds for higher-order functions and polynomial bounds that depend on user-defined inductive types. Moreover, the analysis handles a limited form of side effects and even outperforms the linear bound inference of previous systems. At the same time, it preserves the expressivity and efficiency of existing AARA techniques. The practicality of the analysis system is demonstrated with an implementation and integration with Inria's OCaml compiler. The implementation is used to automatically derive resource bounds for 411 functions and 6018 lines of code derived from OCaml libraries, the CompCert compiler, and implementations of textbook algorithms. In a case study, the system infers bounds on the number of queries that are sent by OCaml programs to DynamoDB, a commercial NoSQL cloud database service.

Computation by interaction for space-bounded functional programming

When programming with sublinear space constraints one often needs to use special implementation techniques even for simple tasks, such as function composition. In this paper, we study how such implementation techniques can be supported in a functional programming language. Our approach is based on modelling computation by interaction using the Int construction of Joyal, Street & Verity. We apply this construction to a term model of a first-order programming language and use the resulting structure to derive the functional programming language intml. Intml can be understood as a programming language simplification of Stratified Bounded Affine Logic. We formulate intml by means of a type system inspired by Baillot & Terui's Dual Light Affine Logic. We show that it captures the complexity classes flogspace and nflogspace. We illustrate its expressiveness by showing how typical graph algorithms, such a test for acyclicity in undirected graphs, can be represented.

Sound and Complete SLD-Resolution for Bilattice-Based Annotated Logic Programs

We introduce the class of normal bilattice-based annotated first-order logic programs (BAPs) and develop declarative and operational semantics for them. In particular, SLD-resolution for these programs is defined and its soundness and completeness established.