Type Inference Research Articles

A Mechanized Proof of a Textbook Type Unification Algorithm

Unification is the core of type inference algorithms for modern functional programming languages, like Haskell and SML. As a first step towards a formalization of a type inference algorithm for such programming languages, we present a formalization in Coq of a type unification algorithm that follows classic algorithms presented in programming language textbooks. We also report on the use of such formalization to build a correct type inference algorithm for the simply typed λ-calculus.

The history of Standard ML

The ML family of strict functional languages, which includes F#, OCaml, and Standard ML, evolved from the Meta Language of the LCF theorem proving system developed by Robin Milner and his research group at the University of Edinburgh in the 1970s. This paper focuses on the history of Standard ML, which plays a central role in this family of languages, as it was the first to include the complete set of features that we now associate with the name “ML” (i.e., polymorphic type inference, datatypes with pattern matching, modules, exceptions, and mutable state). Standard ML, and the ML family of languages, have had enormous influence on the world of programming language design and theory. ML is the foremost exemplar of a functional programming language with strict evaluation (call-by-value) and static typing. The use of parametric polymorphism in its type system, together with the automatic inference of such types, has influenced a wide variety of modern languages (where polymorphism is often referred to as generics ). It has popularized the idea of datatypes with associated case analysis by pattern matching. The module system of Standard ML extends the notion of type-level parameterization to large-scale programming with the notion of parametric modules, or functors . Standard ML also set a precedent by being a language whose design included a formal definition with an associated metatheory of mathematical proofs (such as soundness of the type system). A formal definition was one of the explicit goals from the beginning of the project. While some previous languages had rigorous definitions, these definitions were not integral to the design process, and the formal part was limited to the language syntax and possibly dynamic semantics or static semantics, but not both. The paper covers the early history of ML, the subsequent efforts to define a standard ML language, and the development of its major features and its formal definition. We also review the impact that the language had on programming-language research.

Symbolic Disintegration with a Variety of Base Measures

Disintegration is a relation on measures and a transformation on probabilistic programs that generalizes density calculation and conditioning, two operations widely used for exact and approximate inference. Existing program transformations that find a disintegration or density automatically are limited to a fixed base measure that is an independent product of Lebesgue and counting measures, so they are of no help in practical cases that require tricky reasoning about other base measures. We present the first disintegrator that handles variable base measures, including discrete-continuous mixtures , dependent products , and disjoint sums . By analogy with type inference, our disintegrator can check a given base measure as well as infer an unknown one that is principal. We derive the disintegrator and prove it sound by equational reasoning from semantic specifications. It succeeds in a variety of applications where disintegration and density calculation had not been previously mechanized.

Epigenetic drivers of Cutaneous T Cell Lymphoma revealed at single cell resolution

Abstract Cutaneous T-cell lymphomas (CTCL) are cancers of mature skin-homing T lymphocytes, but the molecular drivers remain poorly understood. We performed whole genome, exome, epigenome, bulk and single cell (sc) transcriptome analyses of serial peripheral blood, skin, and lymph node biopsies from 20 CTCL patients. Initial studies defined markers of therapy resistance and disease progression in pathways that mediate cell migration, cytokine signaling, and epigenetic regulation. Several were novel to CTCL, including BIRC5 (anti-apoptotic); RRM2 (cell cycle); CXCR4 and LAIR2 (migration). On average 7413 cells from 16 serial samples (>100,000 cells total) passed quality thresholds for scRNA- + TCR-seq. We used Seurat, dittoSeq, and SingleR for quality control, dimensionality reduction (TSNE/UMAP), clustering, marker detection, cell type inference, and visualizations, and confirmed the therapy-resistant expression patterns we previously identified, including LAIR2. TCR clonotypes confirmed patient-specific malignant cell populations and defined clonal changes in serial samples. Pseudotime analysis of serial samples with Slingshot identified significant changes in expression related to changes in therapy, particularly in chemokine, TNF, and NFKB signaling. We identified mutations in STAT3&5 in 3/10 and in epigenetic modifiers (KMT2C/D and DNMT3A) in 6/10 patients. Loss of DNMT3A perturbs T cell differentiation and causes leukemia/lymphoma in mouse models. DNMT3A deficiency and additional epigenetic alterations may be previously unrecognized “drivers” of CTCL. These studies define mechanisms of therapy resistance, map clonal evolution, and reveal novel mechanisms of pathogenesis and progression in CTCL.

Verification of high-level transformations with inductive refinement types

High-level transformation languages like Rascal include expressive features for manipulating large abstract syntax trees: first-class traversals, expressive pattern matching, backtracking and generalized iterators. We present the design and implementation of an abstract interpretation tool, Rabit, for verifying inductive type and shape properties for transformations written in such languages. We describe how to perform abstract interpretation based on operational semantics, specifically focusing on the challenges arising when analyzing the expressive traversals and pattern matching. Finally, we evaluate Rabit on a series of transformations (normalization, desugaring, refactoring, code generators, type inference, etc.) showing that we can effectively verify stated properties.

Research progress of large-scale knowledge graph completion technology

With the continued growth of various knowledge graphs, such as Google Knowledge Map, DBpedia, Microsoft Concept Graph, and YAGO, the knowledge representation system, constructed based on RDF, has become more well-known. The RDF triple format has become the basic description of knowledge in the real world. Due to its simple structure and clear logic, it is easy to understand and implement. Nevertheless, when faced with extremely complicated knowledge and common sense, complete knowledge can become difficult to describe. The construction process of knowledge graphs is bound to lead to incomplete knowledge contained in the graphs. At this point, the knowledge-based completion technology is particularly important for managing such situations. Any existing knowledge graph must be improved continuously through completion technology and newly inferred knowledge. Beginning with the construction of a knowledge graph, this paper divides the problem of knowledge graph completion into two levels: concept completion and instance completion. (1) The concept completion level primarily focuses on the completion of entity types. It is described in terms of three development stages: a logical reasoning mechanism, based on description logic, a type inference mechanism, based on traditional machine learning, and a type inference mechanism, based on representation learning. (2) The instance completion level can be further divided into an RDF triple completion and new instance discovery. This paper focuses on RDF triples completion learning, which includes entity completion or relationship completion and is described in three development stages, such as statistical relational learning, probability learning based on random walks, and knowledge representation learning. Through the review and discussion of the research process, the development status, and the latest progress in the above-mentioned large-scale knowledge graph completion, we present the challenges that the technology will face and the development prospects of future work.

Backward type inference for XML queries

Although XQuery is a statically typed, functional query language for XML data, some of its features such as upward and horizontal XPath axes are typed imprecisely. The main reason is that while the XQuery data model allows us to navigate upwards and between siblings from a given XML node, the type model, e.g., regular tree types, can describe only the subtree structure of the given node. To alleviate this limitation, precise forward type inference systems for XQuery were recently proposed using an extended regular type language that can describe not only a given XML node but also its context. In this paper, as a different approach, we propose a novel backward type inference system for XQuery, based on a type language extended with logical formulas. Our backward type inference system provides an exact typing result for XPath axes and a sound typing result for XQuery expressions.

Knowledge graph entity typing via learning connecting embeddings

Knowledge graph (KG) entity typing aims at inferring possible missing entity type instances in KG, which is a very significant but still under-explored subtask of knowledge graph completion. In this paper, we propose a novel approach for KG entity typing which is trained by jointly utilizing local typing knowledge from existing entity type assertions and global triple knowledge in KGs. Specifically, we present two distinct knowledge-driven effective mechanisms of entity type inference. Accordingly, we build two novel embedding models to realize the mechanisms. Afterward, a joint model via connecting them is used to infer missing entity type instances, which favors inferences that agree with both entity type instances and triple knowledge in KGs. Experimental results on two real-world datasets (Freebase and YAGO) demonstrate the effectiveness of our proposed mechanisms and models for improving KG entity typing.

Ptype: probabilistic type inference

Type inference refers to the task of inferring the data type of a given column of data. Current approaches often fail when data contains missing data and anomalies, which are found commonly in real-world data sets. In this paper, we propose ptype, a probabilistic robust type inference method that allows us to detect such entries, and infer data types. We further show that the proposed method outperforms existing methods.

RETrace: simultaneous retrospective lineage tracing and methylation profiling of single cells.

Retrospective lineage tracing harnesses naturally occurring mutations in cells to elucidate single cell development. Common single-cell phylogenetic fate mapping methods have utilized highly mutable microsatellite loci found within the human genome. Such methods were limited by the introduction of in vitro noise through polymerase slippage inherent in DNA amplification, which we characterized to be approximately 10–100× higher than the in vivo replication mutation rate. Here, we present RETrace, a method for simultaneously capturing both microsatellites and methylation-informative cytosines to characterize both lineage and cell type, respectively, from the same single cell. An important unique feature of RETrace was the introduction of linear amplification of microsatellites in order to reduce in vitro amplification noise. We further coupled microsatellite capture with single-cell reduced representation bisulfite sequencing (scRRBS), to measure the CpG methylation status on the same cell for cell type inference. When compared to existing retrospective lineage tracing methods, RETrace achieved higher accuracy (88% triplet accuracy from an ex vivo HCT116 tree) at a higher cell division resolution (lowering the required number of cell division difference between single cells by approximately 100 divisions). Simultaneously, RETrace demonstrated the ability to capture on average 150,000 unique CpGs per single cell in order to accurately determine cell type. We further formulated additional developments that would allow high-resolution mapping on microsatellite-stable cells or tissues with RETrace. Overall, we present RETrace as a foundation for multi-omics lineage mapping and cell typing of single cells.

Fuzzy Expert System for Prediction of Prostate Cancer

A fuzzy expert system (FES) for the prediction of prostate cancer (PC) is prescribed in this paper. Age, prostate-specific antigen (PSA), prostate volume (PV) and [Formula: see text] Free PSA ([Formula: see text]FPSA) are fed as inputs into the FES and prostate cancer risk (PCR) is obtained as the output. Using knowledge-based rules in Mamdani type inference method the output is calculated. If PCR [Formula: see text], then the patient shall be advised to go for a biopsy test for confirmation. The efficacy of the designed FES is tested against a clinical dataset. The true prediction for all the patients turns out to be [Formula: see text] whereas only for positive biopsy cases it rises to [Formula: see text]. This simple yet effective FES can be used as supportive tool for decision-making in medical diagnosis.

A computational complexity analysis of tunable type inference for Generic Universe Types

We address questions related to the computational complexity of inferring types for a particular type system, Generic Universe Types (GUT) [1,2], for Java programs. GUT is useful for many applications, such as program verification [3], thread synchronization, and memory management. However, requiring the programmer to explicitly provide type information is onerous, which motivates the problem of automatically inferring types. In contrast to classical type systems, ownership type systems such as GUT may have multiple typings that satisfy the type system's rules. It is therefore appropriate for the inference to be tunable — that is, the programmer can indicate preferences for certain typings via breakable constraints and/or by partial annotations. A question then is whether efficient algorithms exist for the type inference problem. In this work we establish the following results for the type inference problem for GUT [2]. (1) The tunable type inference problem that allows breakable constraints is NP-hard, (2) an encoding of the problem as boolean satisfiability (SAT), as in prior work, is indeed a polynomial-time reduction, (3) P ≠ NP implies that the problem is not approximable in polynomial time within an approximation ratio of n1−ϵ for any ϵ>0, and (4) while some restricted versions of the problem of practical interest, such as when breakable constraints are forbidden, are in P, others remain NP-hard. Our results justify the prior approach to the problem that is based on reduction to SAT. Apart from these results, given the observation in prior work that instances of the problem that arise in practice appear to be easy, we address the natural question as to what hard instances may look like, and whether they may arise in practice. We identify a class of hard instances of the problem by devising a method to generate such instances starting at instances of the Vertex Cover problem, which is known to be NP-hard. We then analyze the structural properties of such instances as compared to easy instances of similar size. We find that for the classes of instances we consider, certain SAT structural parameters may be predictive of empirical hardness.

Extending Liquid Types to Arrays

A liquid type is an ordinary Hindley-Milner type annotated with a logical predicate that states the properties satisfied by the elements of that type. Liquid types are a powerful tool for program verification, as programmers can use them to specify pre- and post conditions of their programs, whereas the predicates of intermediate variables and auxiliary functions are inferred automatically. Type inference is feasible in this context, as the logical predicates within liquid types are constrained to a quantifier-free logic to maintain decidability. In this article, we extend liquid types by allowing them to contain quantified properties on arrays so that they can be used to infer invariants on array-related programs (e.g., implementations of sorting algorithms). Although quantified logic is, in general, undecidable, we restrict properties on arrays to a decidable subset introduced by Bradley et al. We describe in detail the extended type system, the verification condition generator, and the iterative weakening algorithm for inferring invariants. After proving the correctness and completeness of these two algorithms, we apply them to find invariants on a set of algorithms involving array manipulations.

Formal Verification of Masking Countermeasures for Arithmetic Programs

Cryptographic algorithms are widely used to protect data privacy in many aspects of daily lives from smart card to cyber-physical systems. Unfortunately, programs implementing cryptographic algorithms may be vulnerable to practical power side-channel attacks, which may infer private data via statistical analysis of the correlation between power consumptions of an electronic device and private data. To thwart these attacks, several masking schemes have been proposed, giving rise to effective countermeasures for reducing the statistical correlation between private data and power consumptions. However, programs that rely on secure masking schemes are not secure a priori. Indeed, designing effective masking programs is a labor intensive and error-prone task. Although some techniques have been proposed for formally verifying masking countermeasures and for quantifying masking strength, they are currently limited to Boolean programs and suffer from low accuracy. In this work, we propose an approach for formally verifying masking countermeasures of arithmetic programs. Our approach is more accurate for arithmetic programs and more scalable for Boolean programs comparing to the existing approaches. It is essentially a synergistic integration of type inference and model-counting based methods, armed with domain specific heuristics. The type inference system allows a fast deduction of leakage-freeness of most intermediate computations, the model-counting based methods accounts for completeness, namely, to eliminate spurious flaws, and the heuristics facilitate both type inference and model-counting based reasoning, which improve scalability and efficiency in practice. In case that the program does contain leakage, we provide a method to quantify its masking strength. A distuiguished feature of our type sytem lies in its support of compositonal reasoning when verifying programs with procedure calls, so the need of inlining procedures can be significantly reduced. We have implemented our methods in a verification tool <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">QMVerif</small> which has been extensively evaluated on cryptographic benchmarks including full AES, DES and MAC-Keccak. The experimental results demonstrate the effectiveness and efficiency of our approach, especially for compositional reasoning. In particular, our tool is able to automatically prove leakage-freeness of arithmetic programs for which only manual proofs exist so far; it is also significantly faster than the state-of-the-art tools: EasyCrypt on common arithmetic programs, <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">QMSInfer</small> , SC Sniffer and maskVerif on Boolean programs.

The Inclusive Synthetic Control Method

The Synthetic Control Method (SCM) estimates the causal effect of a policy intervention in a panel data setting with only a few treated units and control units. The treated outcome in the absence of the intervention is recovered by a weighted average of the control units. The latter cannot be affected by the intervention, neither directly nor indirectly. We introduce the inclusive synthetic control method (iSCM), a novel and intuitive synthetic control modification that allows including units potentially affected directly or indirectly by an intervention in the donor pool. Our method is well suited for applications with multiple treated units where including treated units in the donor pool substantially improves the pre-intervention fit and/or for applications where some of the units in the donor pool might be affected by spillover effects. Our iSCM is very easy to implement, and any synthetic control type estimation and inference procedure can be used. Finally, as an illustrative empirical example, we re-estimate the causal effect of German reunification on GDP per capita allowing for spillover effects from West Germany to Austria.

Pointer life cycle types for lock-free data structures with memory reclamation

We consider the verification of lock-free data structures that manually manage their memory with the help of a safe memory reclamation (SMR) algorithm. Our first contribution is a type system that checks whether a program properly manages its memory. If the type check succeeds, it is safe to ignore the SMR algorithm and consider the program under garbage collection. Intuitively, our types track the protection of pointers as guaranteed by the SMR algorithm. There are two design decisions. The type system does not track any shape information, which makes it extremely lightweight. Instead, we rely on invariant annotations that postulate a protection by the SMR. To this end, we introduce angels, ghost variables with an angelic semantics. Moreover, the SMR algorithm is not hard-coded but a parameter of the type system definition. To achieve this, we rely on a recent specification language for SMR algorithms. Our second contribution is to automate the type inference and the invariant check. For the type inference, we show a quadratic-time algorithm. For the invariant check, we give a source-to-source translation that links our programs to off-the-shelf verification tools. It compiles away the angelic semantics. This allows us to infer appropriate annotations automatically in a guess-and-check manner. To demonstrate the effectiveness of our type-based verification approach, we check linearizability for various list and set implementations from the literature with both hazard pointers and epoch-based memory reclamation. For many of the examples, this is the first time they are verified automatically. For the ones where there is a competitor, we obtain a speed-up of up to two orders of magnitude.

Kind inference for datatypes

In recent years, languages like Haskell have seen a dramatic surge of new features that significantly extends the expressive power of their type systems. With these features, the challenge of kind inference for datatype declarations has presented itself and become a worthy research problem on its own. This paper studies kind inference for datatypes. Inspired by previous research on type-inference, we offer declarative specifications for what datatype declarations should be accepted, both for Haskell98 and for a more advanced system we call PolyKinds, based on the extensions in modern Haskell, including a limited form of dependent types. We believe these formulations to be novel and without precedent, even for Haskell98. These specifications are complemented with implementable algorithmic versions. We study soundness, completeness and the existence of principal kinds in these systems, proving the properties where they hold. This work can serve as a guide both to language designers who wish to formalize their datatype declarations and also to implementors keen to have principled inference of principal types.

Non-Employment Activity Type Imputation from Points of Interest and Mobility Data at an Individual Level: How Accurate Can We Get?

Human activity type inference has long been the focus for applications ranging from managing transportation demand to monitoring changes in land use patterns. Today’s ever increasing volume of mobility data allow researchers to explore a wide range of methodological approaches for this task. Such data, however, lack reference observations that would allow the validation of methodological approaches. This research proposes a methodological framework for urban activity type inference using a Dirichlet multinomial dynamic Bayesian network with an empirical Bayes prior that can be applied to mobility data of low spatiotemporal resolution. The method was validated using open source Foursquare data under different isochrone configurations. The results provide evidence of the limits of activity detection accuracy using such data as determined by the Area Under Receiving Operating Curve (AUROC), log-loss, and accuracy metrics. At the same time, results demonstrate that a hierarchical modeling framework can provide some flexibility against the challenges related to the nature of unsupervised activity classification using trajectory variables and POIs as input.

Polymorphic computation systems: Theory and practice of confluence with call-by-value

We present a new framework of polymorphic computation rules that can accommodate a distinction between values and non-values. It is suitable for analysing fundamental calculi of programming languages. We develop a type inference algorithm and new criteria to check the confluence property. These techniques are then implemented in our automated confluence checking tool PolySOL . Its effectiveness is demonstrated through examination of various calculi, including the call-by-need lambda-calculus, Moggi's computational lambda-calculus, and skew-monoidal categories. • A framework of polymorphic second-order computation with value/non-value distinctions. • A type reconstruction algorithm for polymorphic second-order computation systems. • A theory of confluence in the call-by-value setting. • Confluence criteria without termination. • An automated confluence checking tool PolySOL and its demonstration.

CoEvil: A Coevolutionary Model for Crime Inference Based on Fuzzy Rough Feature Selection

Millions of crimes arise each year, which threatens public safety and harms the victims. Precise crime inference is of great significance in preventing crimes. The sharply increasing large-scale heterogeneous data provide a chance to reveal the patterns and trends in crimes. Several approaches employing feature-based regression or spatiotemporal distribution fitting are proposed but lack of some considerations: 1) ignore the dynamic mutual influences among crimes and locations; and 2) overlook the large scale, incompleteness, uncertainty, and vagueness in the heterogeneous data. This article comprehensively investigates the reliability and applicability of proposing a coevolutionary model to formulate the interaction pattern among the crimes and locations and develops a fuzzy-rough-set-based feature selection method to discover the distinctiveness and permanence properties of the crimes and locations with different latent features. Extensive experiments show that our algorithm achieves the mean absolute error of 1.529 (hour) in the crime time inference and the accuracy of 0.653 and 0.633 in the crime type and location inference, which surpass the state of the arts more than 6.5, 1.9, and 1.8 times, respectively. Additional experiments on different parameter settings of our model are provided to further explore its effectiveness and scalability.