Proof Attempts Research Articles

A failed proof can yield a useful test

AbstractA successful automated program proof is, in software verification, the ultimate triumph. In practice, however, the road to such success is paved with many failed proof attempts. Unlike a failed test, which provides concrete evidence of an actual bug in the program, a failed proof leaves the programmer in the dark. Can we instead learn something useful from it? The work reported here takes advantage of the rich information that some automatic provers internally collect about the program when attempting a proof. If the proof fails, the Proof2Test tool presented in this article uses the counterexample generated by the prover (specifically, the SMT solver underlying the Boogie tool used in the AutoProof system to perform correctness proofs of contract‐equipped Eiffel programs) to produce a failed test, which provides the programmer with immediately exploitable information to correct the program. The discussion presents Proof2Test and the application of the ideas and tool to a collection of representative examples.

Strongly-Normalizing Higher-Order Relational Queries

Language-integrated query is a powerful programming construct allowing database queries and ordinary program code to interoperate seamlessly and safely. Language-integrated query techniques rely on classical results about the nested relational calculus, stating that its queries can be algorithmically translated to SQL, as long as their result type is a flat relation. Cooper and others advocated higher-order nested relational calculi as a basis for language-integrated queries in functional languages such as Links and F#. However, the translation of higher-order relational queries to SQL relies on a rewrite system for which no strong normalization proof has been published: a previous proof attempt does not deal correctly with rewrite rules that duplicate subterms. This paper fills the gap in the literature, explaining the difficulty with a previous proof attempt, and showing how to extend the $\top\top$-lifting approach of Lindley and Stark to accommodate duplicating rewrites. We also show how to extend the proof to a recently-introduced calculus for heterogeneous queries mixing set and multiset semantics.

Uniform labelled calculi for preferential conditional logics based on neighbourhood semantics

Abstract The preferential conditional logic $ \mathbb{PCL} $, introduced by Burgess, and its extensions are studied. First, a natural semantics based on neighbourhood models, which generalizes Lewis’ sphere models for counterfactual logics, is proposed. Soundness and completeness of $ \mathbb{PCL} $ and its extensions with respect to this class of models are proved directly. Labelled sequent calculi for all logics of the family are then introduced. The calculi are modular and have standard proof-theoretical properties, the most important of which is admissibility of cut that entails a syntactic proof of completeness of the calculi. By adopting a general strategy, root-first proof search terminates, thereby providing a decision procedure for $ \mathbb{PCL} $ and its extensions. Finally, semantic completeness of the calculi is established: from a finite branch in a failed proof attempt it is possible to extract a finite countermodel of the root sequent. The latter result gives a constructive proof of the finite model property of all the logics considered.

Ground confluence of order-sorted conditional specifications modulo axioms

Terminating functional programs should be deterministic, i.e., should evaluate to a unique result, regardless of the evaluation order. For equational functional programs such determinism is exactly captured by the ground confluence property. For operationally terminating conditional equations this is equivalent to ground local confluence, which follows from local confluence. Checking local confluence by computing critical pairs is the standard way to check ground confluence [33]. The problem is that some perfectly reasonable equational programs are not locally confluent and it can be very hard or even impossible to make them so by adding more equations. We propose three methods, called Methods 1–3, that can be synergistically combined to prove an order-sorted conditional specification modulo axioms B ground locally confluent. Method 1 applies the strategy proposed in [14] to use non-joinable critical pairs as completion hints to either achieve local confluence or reduce the number of critical pairs. Method 2 uses the inductive joinability inference system proposed in this paper to try to prove the critical pairs remaining after applying Method 1 ground joinable. It can furthermore show ground local confluence of the original specification. Method 3 is hierarchical in nature: it can be used to prove the ground local confluence of a conditional equational specification whose conditions belong to a subspecification that has already been proved ground confluent and operationally terminating, and that is conservatively extended by the overall specification in an appropriate sense. These methods apply to order-sorted and possibly conditional equational programs modulo axioms such as, e.g., Maude functional modules. We show their effectiveness in proving the ground confluence of non-trivial examples that have eluded previous proof attempts.

FLAWS IN PROOF CONSTRUCTIONS OF POSTGRADUATE MATHEMATICS EDUCATION STUDENT TEACHERS

Intending to improve the teaching and learning of the notion of mathematical proof this study seeks to uncover the kinds of flaws in postgraduate mathematics education student teachers. Twenty-three student teachers responded to a proof task involving the concepts of transposition and multiplication of matrices. Analytic induction strategy that drew ideas from the literature on evaluating students’ proof understanding and Yang and Lin’s model of proof comprehension applied to informants’ written responses to detect the kinds of flaws in postgraduates’ proof attempts. The study revealed that the use of empirical verifications was dominant and in situations. Whereby participants attempted to argue using arbitrary mathematical objects, the cases considered did not represent the most general case. Flawed conceptualizations uncovered by this study can contribute to efforts directed towards fostering strong subject content command among school mathematics teachers.

Verification Templates for the Analysis of User Interface Software Design

The paper describes templates for model-based analysis of usability and safety aspects of user interface software design. The templates crystallize general usability principles commonly addressed in user-centred safety requirements, such as the ability to undo user actions, the visibility of operational modes, and the predictability of user interface behavior. These requirements have standard forms across different application domains, and can be instantiated as properties of specific devices. The modeling and analysis process is carried out using the Prototype Verification System (PVS), and is further facilitated by structuring the specification of the device using a format that is designed to be generic across interactive systems. A concrete case study based on a commercial infusion pump is used to illustrate the approach. A detailed presentation of the automated verification process using PVS shows how failed proof attempts provide precise information about problematic user interface software features.

Understanding Counterexamples for Relational Properties with DIbugger

Software verification is a tedious process that involves the analysis of multiple failed verification attempts, and adjustments of the program or specification. This is especially the case for complex requirements, e.g., regarding security or fairness, when one needs to compare multiple related runs of the same software. Verification tools often provide counterexamples consisting of program inputs when a proof attempt fails, however it is often not clear why the reported counterexample leads to a violation of the checked property. In this paper, we enhance this aspect of the software verification process by providing DIbugger, a tool for analyzing counterexamples of relational properties, allowing the user to debug multiple related programs simultaneously.

What should be the object of research with respect to the notion of mathematical proof?

Despite its central place in the mathematics curriculum the notion mathematical proof has failed to permeate the curriculum at all scholastic levels. While the concept of mathematical proof can serve as a vehicle for mathematical thinking, studies have revealed that students experience serious difficulties with proving that include: not knowing how to begin the proving process, the proclivity to use empirical verifications for tasks that call for axiomatic methods of proving, resorting to rote memorization of uncoordinated fragments of proof facts. While several studies have been conducted with the aim of addressing students’ fragile grasp of mathematical proof the majority of such studies have been based on activities that involve students reflecting and expressing their level of convincement in arguments supplied by the researchers thereby compromising the voice of the informants. Further, research focus has been on the front instead of the back of mathematics. Hence, there is a dearth in research studies into students’ thinking processes around mathematical proof that are grounded in students’ own proof attempts. Therefore current research strides should aim at identifying critical elements of students’ knowledge of the notion of proof that should be informed by students’ actual individual proof construction attempts.

Automating Event-B invariant proofs by rippling and proof patching

The use of formal method techniques can contribute to the production of more reliable and dependable systems. However, a common bottleneck for industrial adoption of such techniques is the needs for interactive proofs. We use a popular formal method, called Event-B, as our working domain, and set invariant preservation (INV) proofs as targets, because INV proofs can account for a significant proportion of the proofs requiring human interactions. We apply an inductive theorem proving technique, called rippling, for Event-B INV proofs. Rippling automates proofs using meta-level guidance. The guidance is in particular useful to develop proof patches to recover failed proof attempts. We are interested in the case when a missing lemma is required. We combine a scheme-based theory-exploration system, called IsaScheme [MRMDB10], with rippling to develop a proof patch via lemma discovery. We also develop two new proof patches to unfold operator definitions and to suggest case-splits, respectively. The combined use of rippling with these three proof patches as a proof method significantly improves the proof automation for our evaluation set.

A polyhedral proof of a wreath product identity

In 2013, Beck and Braun proved and generalized multiple identities involving permutation statistics via discrete geometry. Namely, they recognized the identities as specializations of integer point transform identities for certain polyhedral cones. They extended many of their proof techniques to obtain identities involving wreath products, but some identities were resistant to their proof attempts. In this article, we provide a geometric justification of one of these wreath product identities, which was first established by Biagioli and Zeng.

Teaching the Formalization of Mathematical Theories and Algorithms via the Automatic Checking of Finite Models

Education in the practical applications of logic and proving such as the formal specification and verification of computer programs is substantially hampered by the fact that most time and effort that is invested in proving is actually wasted in vain: because of errors in the specifications respectively algorithms that students have developed, their proof attempts are often pointless (because the proposition proved is actually not of interest) or a priori doomed to fail (because the proposition to be proved does actually not hold), this is a frequent source of frustration and gives formal methods a bad reputation. RISCAL (RISC Algorithm Language) is a formal specification language and associated software system that attempts to overcome this problem by making logic formalization fun rather than a burden. To this end, RISCAL allows students to easily validate the correctness of instances of propositions respectively algorithms by automatically evaluating/executing and checking them on (small) finite models. Thus many/most errors can be quickly detected and subsequent proof attempts can be focused on propositions that are more/most likely to be both meaningful and true.

Proof assisted bounded and unbounded symbolic model checking of software and system models

We have implemented various symbolic model checking algorithms, such as BMC, k-Induction and IC3 for B, Event-B and other modeling languages. The high-level nature of software models accounts for complicated constraints arising in these symbolic analysis techniques. In this article we suggest using static information stemming from proof obligations to simplify occurring constraints. We show how to include proof information in the aforementioned algorithms. Using different benchmarks we compare explicit state to symbolic model checking as well as techniques with and without proof assistance. In particular for models with large branching factor, e.g., due to complicated data values being manipulated, the symbolic techniques fare much better than explicit state model checking. The inclusion of proof information results in further performance improvements. • We implemented bounded model checking, k-Induction and IC3 for B and Event-B. • We show how to use static information from proof attempts to improve performance. • We benchmark against ProB's explicit state model checker. • We discuss applicability to other formalisms such as TLA + and Z.

Designing a Web-based Learning Support System for Flow-chart Proving in School Geometry

As international research confirms, many secondary school students can find it difficult to construct mathematical proofs. In this article, we explain the pedagogical and technological underpinnings of a web-based learning support system for students who are just starting to tackle deductive proving in geometry. We show how the system was designed to enable students to access the study of proofs in geometry by tackling proof problems where they can ‘drag’ sides, angles and triangles from the figural diagram of the problem to on-screen cells within a flow-chart proof format. When doing so, the system automatically converts the figural elements to their symbolic form and identifies any of four kinds of errors in the learners’ proof attempts, providing relevant feedback on-screen. We use empirical examples from our pilot studies to illustrate how this combination of technological features and systematic feedback can support student understanding of the structure of proof and how to plan one. Finally, we point out some limitations to mathematical expression and the usage of the flow-chart format, and indicate the prospect of minimizing such limitations by adopting a learning progression for the introductory lessons concerning deductive proofs.

Equivalence Checking of Two Functional Programs Using Inductive Theorem Provers

In this paper, we present a new method for generating inductive hypothesis for proving an equivalence between two imperative programs: one recursive and one iterative. The basic premise is to convert both programs in the form of term rewriting system that is acceptable to inductive theorem provers and to generate inductive generalization lemmas that such a theorem prover can discharge. As finding inductive generalizations automatically is an undecidable problem, we will focus on a certain class of recursive programs that cannot be handled by existing tools. We develop criteria under which we can prove the equivalence of two programs. A technique based on schematic programs for proving concrete ones is proposed.Schematic generalization lemmas are computed directly from given specifications.Concrete lemmas are obtained by instantiation of schematic generalization lemmas.The technique is applied prior to any proof attempt, then save time.Only the validity of the schematic generalization lemma is checked.

Testing noninterference, quickly

AbstractInformation-flow control mechanisms are difficult both to design and to prove correct. To reduce the time wasted on doomed proof attempts due to broken definitions, we advocate modern random-testing techniques for finding counterexamples during the design process. We show how to use QuickCheck, a property-based random-testing tool, to guide the design of increasingly complex information-flow abstract machines, leading up to a sophisticated register machine with a novel and highly permissive flow-sensitive dynamic enforcement mechanism that is sound in the presence of first-class public labels. We find that both sophisticated strategies for generating well-distributed random programs and readily falsifiable formulations of noninterference properties are critically important for efficient testing. We propose several approaches and evaluate their effectiveness on a collection of injected bugs of varying subtlety. We also present an effective technique for shrinking large counterexamples to minimal, easily comprehensible ones. Taken together, our best methods enable us to quickly and automatically generate simple counterexamples for more than 45 bugs. Moreover, we show how testing guides the discovery of the sophisticated invariants needed for the noninterference proof of our most complex machine.

Machine Learning for First-Order Theorem Proving

We applied two state-of-the-art machine learning techniques to the problem of selecting a good heuristic in a first-order theorem prover. Our aim was to demonstrate that sufficient information is available from simple feature measurements of a conjecture and axioms to determine a good choice of heuristic, and that the choice process can be automatically learned. Selecting from a set of 5 heuristics, the learned results are better than any single heuristic. The same results are also comparable to the prover's own heuristic selection method, which has access to 82 heuristics including the 5 used by our method, and which required additional human expertise to guide its design. One version of our system is able to decline proof attempts. This achieves a significant reduction in total time required, while at the same time causing only a moderate reduction in the number of theorems proved. To our knowledge no earlier system has had this capability.

Testing noninterference, quickly

Information-flow control mechanisms are difficult to design and labor intensive to prove correct. To reduce the time wasted on proof attempts doomed to fail due to broken definitions, we advocate modern random testing techniques for finding counterexamples during the design process. We show how to use QuickCheck, a property-based random-testing tool, to guide the design of a simple information-flow abstract machine. We find that both sophisticated strategies for generating well-distributed random programs and readily falsifiable formulations of noninterference properties are critically important. We propose several approaches and evaluate their effectiveness on a collection of injected bugs of varying subtlety. We also present an effective technique for shrinking large counterexamples to minimal, easily comprehensible ones. Taken together, our best methods enable us to quickly and automatically generate simple counterexamples for all these bugs.

Granularity Analysis for Mathematical Proofs

Mathematical proofs generally allow for various levels of detail and conciseness, such that they can be adapted for a particular audience or purpose. Using automated reasoning approaches for teaching proof construction in mathematics presupposes that the step size of proofs in such a system is appropriate within the teaching context. This work proposes a framework that supports the granularity analysis of mathematical proofs, to be used in the automated assessment of students' proof attempts and for the presentation of hints and solutions at a suitable pace. Models for granularity are represented by classifiers, which can be generated by hand or inferred from a corpus of sample judgments via machine-learning techniques. This latter procedure is studied by modeling granularity judgments from four experts. The results provide support for the granularity of assertion-level proofs but also illustrate a degree of subjectivity in assessing step size.

On the meaning of logical completeness

Goedel's completeness theorem is concerned with provability, while Girard's theorem in ludics (as well as full completeness theorems in game semantics) are concerned with proofs. Our purpose is to look for a connection between these two disciplines. Following a previous work [3], we consider an extension of the original ludics with contraction and universal nondeterminism, which play dual roles, in order to capture a polarized fragment of linear logic and thus a constructive variant of classical propositional logic. We then prove a completeness theorem for proofs in this extended setting: for any behaviour (formula) A and any design (proof attempt) P, either P is a proof of A or there is a model M of the orthogonal of A which defeats P. Compared with proofs of full completeness in game semantics, ours exhibits a striking similarity with proofs of Goedel's completeness, in that it explicitly constructs a countermodel essentially using Koenig's lemma, proceeds by induction on formulas, and implies an analogue of Loewenheim-Skolem theorem.

Automated Flaw Detection in Algebraic Specifications

In interactive theorem proving practice a significant amount of time is spent on unsuccessful proof attempts of wrong conjectures. An automatic method that reveals them by generating finite counter examples would offer an extremely valuable support for a proof engineer by saving his time and effort. In practice, such counter examples tend to be small, so usually there is no need to search for big instances. Most definitions of functions or predicates on infinite structures do not preserve the semantics if a transition to arbitrary finite substructures is made. We propose constraints which guarantee a correct axiomatization on finite structures and present an approach which uses the Alloy Analyzer to generate finite instances of theories in the theorem prover KIV. It is evaluated on the library of basic data types as well as on some challenging case studies in KIV. The technique is implemented using the Kodkod constraint solver which is a successor of Alloy.