Learnability Results Research Articles

PARLE-G: Provable Automated Representation and Analysis Framework for Learnability Evaluation of Generic PUF Compositions

Besides enormous research efforts in the design of Physically Unclonable Functions (PUFs), its vulnerabilities are still being exploited using machine learning (ML) based model-building attacks. Due to inherent complicacy in exploring and manually converging to a strong PUF composition, the challenge of building ML-attack resistant PUFs continues. Hence, it becomes imperative to develop an automated framework that can formally assess the learnability of different PUF constructions and compositions to guide the designer to explore resilient PUFs. In this work, we present an automated analysis framework (PARLE-G), to formally represent and evaluate the Probably Approximately Correct (PAC) learnability of PUF constructions and their compositions. A high-level specification language PUF-G has been developed to structurally represent any PUF composition comprising a specified set of primitive components and composition operations. The tool takes a PUF design represented in PUF-G language as input and returns its PAC learnability result, identifying a suitable PAC learning algorithm and the PAC model parameters based on the input PUF design. PUF designs proven to be learnable by PARLE-G are segregated into different classes based on the asymptotic complexity of their learnability bounds. Such automated analysis helps a designer to make informed design choices, thereby strengthening a PUF construction from the architectural level.

PAC Learning and Stabilizing Hedonic Games: Towards a Unifying Approach.

We study PAC learnability and PAC stabilizability of Hedonic Games (HGs), i.e., efficiently inferring preferences or core-stable partitions from samples. We first expand the known learnability/stabilizability landscape for some of the most prominent HGs classes, providing results for Friends and Enemies Games, Bottom Responsive, and Anonymous HGs. Then, having a broader view in mind, we attempt to shed light on the structural properties leading to learnability/stabilizability, or lack thereof, for specific HGs classes. Along this path, we focus on the fully expressive Hedonic Coalition Nets representation of HGs. We identify two sets of conditions that lead to efficient learnability, and which encompass all of the known positive learnability results. On the side of stability, we reveal that, while the freedom of choosing an ad hoc adversarial distribution is the most obvious hurdle to achieving PAC stability, it is not the only one. First, we show a distribution independent necessary condition for PAC stability. Then, we focus on W-games, where players have individual preferences over other players and evaluate coalitions based on the least preferred member. We prove that these games are PAC stabilizable under the class of bounded distributions, which assign positive probability mass to all coalitions. Finally, we discuss why such a result is not easily extendable to other HGs classes even in this promising scenario. Namely, we establish a purely computational property necessary for achieving PAC stability.

Learning Set Functions with Limited Complementarity

We study PMAC-learning of real-valued set functions with limited complementarity. We prove, to our knowledge, the first nontrivial learnability result for set functions exhibiting complementarity, generalizing Balcan and Harvey’s result for submodular functions. We prove a nearly matching information theoretical lower bound on the number of samples required, complementing our learnability result. We conduct numerical simulations to show that our algorithm is likely to perform well in practice.

Projective DNF formulae and their revision

Combinatorial and learnability results are proven for projective disjunctive normal forms, a class of DNF expressions introduced by Valiant.

K-Valued non-associative Lambek grammars are learnable from generalized functor-argument structures

This paper is concerned with learning categorial grammars from positive examples in the model of Gold. Functor-argument structures (written FA) are usual syntactical decompositions of sentences in sub-components distinguishing the functional parts from the argument parts defined in the case of classical categorial grammars also known as AB-grammars. In the case of non-associative type-logical grammars, we propose a similar notion that we call generalized functor-argument structures and we show that these structures capture the essence of non-associative Lambek (NL) calculus without product. We show that (i) rigid and k -valued non-associative Lambek (NL without product) grammars are learnable from generalized functor-argument structured sentences. We also define subclasses of k -valued grammars in terms of arity. We first show that (ii) for each k and each bound on arity the class of FA-arity bounded k -valued NL languages of FA structures is finite and (iii) that FA-arity bounded k -valued NL grammars are learnable both from strings and from FA structures as a corollary. Result (i) is obtained from (ii); this learnability result (i) is interesting and surprising when compared to other results: in fact we also show that (iv) this class has infinite elasticity. Moreover, these classes are very close to classes like rigid associative Lambek grammars learned from natural deduction structured sentences (that are different and much richer than FA or generalized FA) or to k -valued non-associative Lambek grammars unlearnable from strings or even from bracketed strings. Thus, the class of k -valued non-associative Lambek grammars learned from generalized functor-argument sentences is at the frontier between learnable and unlearnable classes of languages.

Control structures in hypothesis spaces: the influence on learning

In any learnability setting, hypotheses are conjectured from some hypothesis space. Studied herein are the influence on learnability of the presence or absence of certain control structures in the hypothesis space. First presented are control structure characterizations of some rather specific but illustrative learnability results. The presence of these control structures is thereby shown essential to maintain full learning power. Then presented are the main theorems. Each of these non-trivially characterizes the invariance of a learning class over hypothesis space V and the presence of a particular projection control structure, called proj , in V as: V has suitable instances of all denotational control structures. In a sense, then, proj epitomizes the control structures whose presence need not help and whose absence need not hinder learning power.

DERANDOMIZED LEARNING OF BOOLEAN FUNCTIONS OVER FINITE ABELIAN GROUPS

We employ the Always Approximately Correct or AAC model defined in [35], to prove learnability results for classes of Boolean functions over arbitrary finite Abelian groups. This model is an extension of Angluin's Query model of exact learning. The Boolean functions we consider belong to approximation classes, i.e. functions that are approximable (in various norms) by few Fourier basis functions, or irreducible characters of the domain Abelian group. We contrast our learnability results to previous results for similar classes in the PAC model of learning with and without membership queries. In addition, we discuss new, natural issues and questions that arise when the AAC model is used. One such question is whether a uniform training set is available for learning any function in a given approximation class. No analogous question seems to have been studied in the context of Angluin's Query model. Another question is whether the training set can be found quickly if the approximation class of the function is completely unknown to the learner, or only partial information about the approximation class is given to the learner (in addition to the answers to membership queries). In order to prove the learnability results in this paper we require new techniques for efficiently sampling Boolean functions using the character theory of finite Abelian groups, as well as the development of algebraic algorithms. The techniques result in other natural applications closely related to learning, for example, query complexity of deterministic algorithms for testing linearity, efficient pseudorandom generators, and estimating VC dimensions for classes of Boolean functions over finite Abelian groups.

PAC Learning under Helpful Distributions

A PAC teaching model -under helpful distributions - is proposed which introduces the classical ideas of teaching models within the PAC setting: a polynomial-sized teaching set is associated with each target concept; the criterion of success is PAC identification; an additional parameter, namely the inverse of the minimum probability assigned to any example in the teaching set, is associated with each distribution; the learning algorithm running time takes this new parameter into account. An Occam razor theorem and its converse are proved. Some classical classes of boolean functions, such as Decision Lists, DNF and CNF formulas are proved learnable in this model. Comparisons with other teaching models are made: learnability in the Goldman and Mathias model implies PAC learnability under helpful distributions. Note that Decision lists and DNF are not known to be learnable in the Goldman and Mathias model. A new PAC model, where simple refers to Kolmogorov complexity, is introduced. We show that most learnability results obtained within previously defined PAC models can be simply derived from more general results in our model.

Elementary Formal Systems, Intrinsic Complexity, and Procrastination

Recently, rich subclasses of elementary formal systems (EFS) have been shown to be identifiable in the limit from only positive data. Examples of these classes are Angluin's pattern languages, unions of pattern languages by Wright and Shinohara, and classes of languages definable by length-bounded elementary formal systems studied by Shinohara. The present paper employs two distinct bodies of abstract studies in the inductive inference literature to analyze the learnability of these concrete classes. The first approach uses constructive ordinals to bound the number of mind changes. ω denotes the first limit ordinal. An ordinal mind change bound of ω means that identification can be carried out by a learner that after examining some element(s) of the language announces an upper bound on the number of mind changes it will make before converging; a bound of ω ·2 means that the learner reserves the right to revise this upper bound once; a bound of ω ·3 means the learner reserves the right to revise this upper bound twice, and so on. A bound of ω 2 means that identification can be carried out by a learner that announces an upper bound on the number of times it may revise its conjectured upper bound on the number of mind changes. It is shown in the present paper that the ordinal mind change complexity for identification of languages formed by unions of up to n pattern languages is ω n . It is also shown that this bound is essential. Similar results are also shown to hold for classes definable by length-bounded elementary formal systems with up to n clauses. The second approach employs reductions to study the intrinsic complexity of learnable classes. It is shown that the class of languages formed by taking unions of up to n +1 pattern languages is a strictly more difficult learning problem than the class of languages formed by the union of up to n pattern languages. It is also shown that a similar hierarchy holds for the bound on the number of clauses in the case of languages definable by length-bounded EFS. In addition to building bridges between three distinct areas of inductive inference, viz., learnability of EFS subclasses, ordinal mind change complexity, and intrinsic complexity, this paper also presents results that relate topological properties of learnable classes to that of intrinsic complexity and ordinal mind change complexity. For example, it is shown that a class that is complete according to the reductions for intrinsic complexity has infinite elasticity. Since EFS languages and their learnability results have counterparts in traditional logic programming, the present paper demonstrates the possibility of using abstract results of inductive inference to gain insights into inductive logic programming.

Case-based representation and learning of pattern languages

Pattern languages seem to suit case-based reasoning particularly well. Therefore, the problem of inductively learning pattern languages is paraphrased in a case-based manner. A careful investigation requires a formal semantics for case bases together with similarity measures in terms of formal languages. Two basic semantics are introduced and investigated. It turns out that representability problems are major obstacles for case-based learnability. Restricting the attention to the so-called proper patterns avoids these representability problems. A couple of learnability results for proper pattern languages are derived both for case-based learning from only positive data and for case-based learning from positive and negative data. Under the so-called competing semantics, we show that the learnability result for positive and negative data can be lifted to the general case of arbitrary patterns. Learning under the standard semantics from positive data is closely related to monotonic language learning.