Superset Queries Research Articles

A performance study of three disk-based structures for indexing and querying frequent itemsets

Frequent itemset mining is an important problem in the data mining area. Extensive efforts have been devoted to developing efficient algorithms for mining frequent itemsets. However, not much attention is paid on managing the large collection of frequent itemsets produced by these algorithms for subsequent analysis and for user exploration. In this paper, we study three structures for indexing and querying frequent itemsets: inverted files, signature files and CFP-tree. The first two structures have been widely used for indexing general set-valued data. We make some modifications to make them more suitable for indexing frequent itemsets. The CFP-tree structure is specially designed for storing frequent itemsets. We add a pruning technique based on length-2 frequent itemsets to make it more efficient for processing superset queries. We study the performance of the three structures in supporting five types of containment queries: exact match, subset/superset search and immediate subset/superset search. Our results show that no structure can outperform other structures for all the five types of queries on all the datasets. CFP-tree shows better overall performance than the other two structures.

Teachability of a Subclass of Simple Deterministic Languages

SUMMARY We show teachability of a subclass of simple deterministic languages. The subclass we define is called stack uniform simple deterministic languages. Teachability is derived by showing the query learning algorithm for this language class. Our learning algorithm uses membership, equivalence and superset queries. Then, it terminates in polynomial time. It is already known that simple deterministic languages are polynomial time query learnable by context-free grammars. In contrast, our algorithm guesses a hypothesis by a stack uniform simple deterministic grammar, thus our result is strict teachability of the subclass of simple deterministic languages. In addition, we discuss parameters of the polynomial for teachability. The “thickness” is an important parameter for parsing and it should be one of parameters to evaluate the time complexity.

Learning of Elementary Formal Systems with Two Clauses Using Queries

An elementary formal system, EFS for short, is a kind of logic program over strings, and regarded as a set of rules to generate a language. For an EFS Γ, the language L(Γ) denotes the set of all strings generated by Γ. We consider a new form of EFS, called a restricted two-clause EFS, and denote by $r{\\cal EF\\!S}$ the set of all restricted two-clause EFSs. Then we study the learnability of $r{\\cal EF\\!S}$ in the exact learning model. The class $r{\\cal EF\\!S}$ contains the class of regular patterns, which is extensively studied in Learning Theory. Let Γ* be a target EFS in $r{\\cal EF\\!S}$ of learning. In the exact learning model, an oracle for superset queries answers “yes” for an input EFS Γ in $r{\\cal EF\\!S}$ if L(Γ) is a superset of L(Γ*), and outputs a string in L(Γ*) - L(Γ), otherwise. An oracle for membership queries answers “yes” for an input string w if w is included in L(Γ*), and answers “no”, otherwise. We show that any EFS in $r{\\cal EF\\!S}$ is exactly identifiable in polynomial time using membership and superset queries. Moreover, for other types of queries, we show that there exists no polynomial time learning algorithm for $r{\\cal EF\\!S}$ by using the queries. This result indicates the hardness of learning the class $r{\\cal EF\\!S}$ in the exact learning model, in general.

The complexity of learning concept classes with polynomial general dimension

The general dimension is a combinatorial measure that characterizes the number of queries needed to learn a concept class. We use this notion to show that any p-evaluatable concept class with polynomial query complexity can be learned in polynomial time with the help of an oracle in the polynomial hierarchy, where the complexity of the required oracle depends on the query-types used by the learning algorithm. In particular, we show that for subset and superset queries an oracle in Σ 3 P suffices. Since the concept class of DNF formulas has polynomial query complexity with respect to subset and superset queries with DNF formulas as hypotheses, it follows that DNF formulas are properly learnable in polynomial time with subset and superset queries and the help of an oracle in Σ 3 P . We also show that the required oracle in our main theorem cannot be replaced by an oracle in a lower level of the polynomial-time hierarchy, unless the hierarchy collapses.

Formal language identification: query learning vs. Gold-style learning

A natural approach towards powerful machine learning systems is to enable options for additional machine/user interactions, for instance by allowing the system to ask queries about the concept to be learned. This motivates the development and analysis of adequate formal learning models. In the present paper, we investigate two different types of query learning models in the context of learning indexable classes of recursive languages: Angluin's original model and a relaxation thereof, called learning with extra queries. In the original model the learner is restricted to query languages belonging to the target class, while in the new model it is allowed to query other languages, too. As usual, the following standard types of queries are considered: superset, subset, equivalence, and membership queries. The learning capabilities of the resulting query learning models are compared to one another and to different versions of Gold-style language learning from only positive data and from positive and negative data (including finite learning, conservative inference, and learning in the limit). A complete picture of the relation of all these models has been elaborated. A couple of interesting differences and similarities between query learning and Gold-style learning have been observed. In particular, query learning with extra superset queries coincides with conservative inference from only positive data. This result documents the naturalness of the new query model.

A New Abstract Combinatorial Dimension for Exact Learning via Queries

We introduce an abstract model of exact learning via queries that can be instantiated to all the query learning models currently in use, while being closer to them than previous unifying attempts. We present a characterization of those Boolean function classes learnable in this abstract model, in terms of a new combinatorial notion that we introduce, the abstract identification dimension. Then we prove that the particularization of our notion to specific known protocols such as equivalence, membership, and membership and equivalence queries results in exactly the same combinatorial notions currently known to characterize learning in these models, such as strong consistency dimension, extended teaching dimension, and certificate size. Our theory thus fully unifies all these characterizations. For models enjoying a specific property that we identify, the notion can be simplified while keeping the same characterizations. From our results we can derive combinatorial characterizations of all those other models for query learning proposed in the literature. We can also obtain the first polynomial-query learning algorithms for specific interesting problems such as learning DNF with proper subset and superset queries.

Signature-based structures for objects with set-valued attributes

Aiming at the efficient retrieval of objects with set-valued attributes, we introduce three variations of a new method in order to satisfy subset and superset queries. Our approach is to combine the advantages of two access methods, that of linear Hashing and of tree-shaped methods, on which other similar methods have been previously reported as well. Performance estimation analytical functions for each particular method are presented, followed by a thorough experimental comparison of all investigated structures, where analytical and experimental results deviate 10% on the average. Finally, the results of this performance evaluation are presented and discussed, clearly showing the superiority of the new methods reaching an improvement of up to 85%.

Learning one-variable pattern languages very efficiently on average, in parallel, and by asking queries

A pattern is a finite string of constant and variable symbols. The language generated by a pattern is the set of all strings of constant symbols which can be obtained from the pattern by substituting non-empty strings for variables. We study the learnability of one-variable pattern languages in the limit with respect to the update time needed for computing a new single hypothesis and the expected total learning time taken until convergence to a correct hypothesis. Our results are as follows. First, we design a consistent and set-driven learner that, using the concept of descriptive patterns, achieves update time O(n 2 logn) , where n is the size of the input sample. The best previously known algorithm for computing descriptive one-variable patterns requires time O(n 4 logn) (cf. Angluin, J. Comput. Systems Sci. 21(1) (1980) 46–62). Second, we give a parallel version of this algorithm that requires time O( logn) and O(n 3/ log n) processors on an EREW-PRAM. Third, using a modified version of the sequential algorithm as a subroutine, we devise a learning algorithm for one-variable patterns whose expected total learning time is O(ℓ 2 logℓ) provided the sample strings are drawn from the target language according to a probability distribution with expected string length ℓ. The probability distribution must be such that strings of equal length have equal probability, but can be arbitrary otherwise. Thus, we establish the first algorithm for learning one-variable pattern languages having an expected total learning time that provably differs from the update time by a constant factor only. Finally, we show how the algorithm for descriptive one-variable patterns can be used for learning one-variable patterns with a polynomial number of superset queries with respect to the one-variable patterns as query language.

The learnability of exclusive-or expansions based on monotone DNF formulas

The learnability of the class of exclusive-or expansions based on monotone DNF formulas is investigated. The class consists of the formulas of the form f=f 1⊕⋯⊕f d , where f 1>⋯>f d are monotone DNF formulas. It is shown that any Boolean function can be represented as a formula in this class, and that the representation in the simplest form is unique. Learning algorithms that learn such formulas using various queries are presented: An algorithm with subset and superset queries and one with membership and equivalence queries are given. The former can learn any formula in the class, while the latter is proved to learn formulas of constant depth, i.e., formulas represented as exclusive-or of a constant number of monotone DNF formulas. In spite of the seemingly strong restriction of the depth being constant, the class of formulas of constant depth includes functions with very high complexity in terms of DNF and CNF representations, so the latter algorithm could learn Boolean functions significantly complex otherwise represented.

Oracles and Queries That Are Sufficient for Exact Learning

We show that the class of all circuits is exactly learnable in randomized expected polynomial time using weak subset and weak superset queries. This is a consequence of the following result which we consider to be of independent interest: circuits are exactly learnable in randomized expected polynomial time with equivalence queries and the aid of anNP-oracle. We also show that circuits are exactly learnable in deterministic polynomial time with equivalence queries and a ∑P3-oracle. The hypothesis class for the above learning algorithms is the class of circuits of larger—but polynomially related—size. Also, the algorithms can be adapted to learn the class of DNF formulas with hypothesis class consisting of depth-3 ∧-∨-∧ formulas (by the work of Angluin this is optimal in the sense that the hypothesis class cannot be reduced to DNF formulas, i.e., depth-2 ∨-∧ formulas). We also investigate the power of anNP-oracle in the context of learning withmembershipqueries. We show that there are deterministic learning algorithms that use membership queries and anNP-oracle to learn: monotone boolean functions in time polynomial in the DNF size and CNF size of the target formula; and the class ofO(logn)-DNF∩O(logn)-CNF formulas in time polynomial inn. We also show that, with anNP-oracle and membership queries, there is a randomized expected polynomial time algorithm that learns any class that is learnable from membership queries with unlimited computational power. Using similar techniques, we show the following both for membership and for equivalence queries (when the hypotheses allowed are precisely the concepts in the class); any class learnable with unbounded ?computational-power is learnable in deterministic polynomial time with a ∑:p5-oracle. Furthermore, we identify the combinatorial properties that completely determine learnability in this information-theoretic sense. Finally we point out a consequence of our result in structural complexity theory showing that if everyNPset has polynomial-size circuits then the polynomial hierarchy collapses to ZPPNP.

Structural analysis of polynomial-time query learnability

In this paper we study the computational power of polynomialtime query learning systems for several query types, and the computational complexity of a “learning problem” for several representation classes. As corollaries of our results, we prove some polynomial-time nonlearnability results, and relate polynomial-time learnability of some representation classes to the complexity of representation finding problems of P/poly oracles. For example, forCIR, a representation class by logical circuits, it is shown that P(NP 1 () ) is an upper bound of power of query learning systems forCIR, and that P(NP 1 () ) is also a lower bounds of power of query learning systems forCIR when they are used to learn a certain subclassR ofCIR. It is also shown that the problem of learningCIR is P(NP(NP 1 () ))-solvable. Then, using these results, the following relations are proved: (1) If, for someA ε P/poly, the representation finding problem ofA is not in P(NP 1 A ), thenCIR is not polynomial-time query learnable even by using queries such as membership, equivalence, subset, superset, etc. (2) On the other hand, if the above-mentioned subclassR ofCIR is not polynomial-time query learnable by using subset and superset queries, then some Bε P/poly exists such that its representation finding problem is not in P(NP 1 B ).