Min-wise Independent Permutations Research Articles

Growth of the perfect sequence covering array number

In this note we answer positively an open question posed by Yuster in 2020 [14] on the polynomial boundedness of the perfect sequence covering array numberg(n,k) (PSCA number). The latter determines the (renormalized) minimum row-count that perfect sequence covering arrays (PSCAs) can possess. PSCAs are matrices with permutations in S_n as rows, such that each ordered k-sequence of distinct elements of [n] is covered by the same number of rows. We obtain the result after illuminating an isomorphism between this structure from design theory and a special case of min-wise independent permutations. Afterwards, we point out that asymptotic bounds and constructions can be transferred between these two structures. Moreover, we sharpen asymptotic lower bounds for g(n, k) and improve upper bounds for g(n, 4) and g(n, 3), for some concrete values of n. We conclude with some open questions and propose a new matrix class being potentially advantageous for searching PSCAs.

Reachability Querying: Can It Be Even Faster?

As an important graph operator, reachability query has been extensively studied over decades, which is to check whether a vertex can reach another vertex over a large directed graph $G$ with $n$ vertices and $m$ edges. The efforts made in the reported studies have greatly improved the query time of answering reachability queries online, while reducing the offline index construction time to construct an index with a reasonable size given the approach taken, where an entry in an index for a vertex is called a label of the vertex. Among all the work, the recent development of IP (Independent Permutation) employs randomness using $k$ -min-wise independent permutations to process reachability queries, and shows the advantages for both query time and index construction time. In this paper, we propose a new Bloom filter Labeling, denoted as BFL. We show that the probability to answer reachability queries by BFL can be bounded, and BFL has high pruning power to answer more reachability queries directly. We give algorithms and analyze the pruning power of BFL. We conduct extensive studies using 19 large datasets. We show that BFL with an interval label performs best in the index construction time for all 19 cases, and performs best in query time for 16 out of 19 cases.

LSH-Preserving Functions and Their Applications

Locality sensitive hashing (LSH) is a key algorithmic tool that is widely used both in theory and practice. An important goal in the study of LSH is to understand which similarity functions admit an LSH, that is, are LSHable. In this article, we focus on the class of transformations such that given any similarity that is LSHable, the transformed similarity will continue to be LSHable. We show a tight characterization of all such LSH-preserving transformations: they are precisely the probability generating functions, up to scaling. As a concrete application of this result, we study which set similarity measures are LSHable. We obtain a complete characterization of similarity measures between two sets A and B that are ratios of two linear functions of ∣ A ∩ B ∣, ∣ A ▵ B ∣, ∣ A ∪ B ∣: such a measure is LSHable if and only if its corresponding distance is a metric. This result generalizes the well-known LSH for the Jaccard set similarity, namely, the minwise-independent permutations, and obtains LSHs for many set similarity measures that are used in practice. Using our main result, we obtain a similar characterization for set similarities involving radicals.

Efficient algorithms for large-scale local triangle counting

In this article, we study the problem of approximate local triangle counting in large graphs. Namely, given a large graph G =( V,E ) we want to estimate as accurately as possible the number of triangles incident to every node v ∈ V in the graph. We consider the question both for undirected and directed graphs. The problem of computing the global number of triangles in a graph has been considered before, but to our knowledge this is the first contribution that addresses the problem of approximate local triangle counting with a focus on the efficiency issues arising in massive graphs and that also considers the directed case. The distribution of the local number of triangles and the related local clustering coefficient can be used in many interesting applications. For example, we show that the measures we compute can help detect the presence of spamming activity in large-scale Web graphs, as well as to provide useful features for content quality assessment in social networks. For computing the local number of triangles (undirected and directed), we propose two approximation algorithms, which are based on the idea of min-wise independent permutations [Broder et al. 1998]. Our algorithms operate in a semi-streaming fashion, using O (| V |) space in main memory and performing O (log | V |) sequential scans over the edges of the graph. The first algorithm we describe in this article also uses O (| E |) space of external memory during computation, while the second algorithm uses only main memory. We present the theoretical analysis as well as experimental results on large graphs, demonstrating the practical efficiency of our approach.

Disjoint bases in a polymatroid

AbstractLet f : 2N → $\cal {Z}$+ be a polymatroid (an integer‐valued non‐decreasing submodular set function with f(∅︁) = 0). We call S ⊆ N a base if f(S) = f(N). We consider the problem of finding a maximum number of disjoint bases; we denote by m* be this base packing number. A simple upper bound on m* is given by k* = max{k : ΣiεNfA(i) ≥ kfA(N),∀A ⊆ N} where fA(S) = f(A ∪ S) ‐ f(A). This upper bound is a natural generalization of the bound for matroids where it is known that m* = k*. For polymatroids, we prove that m* ≥ (1 − o(1))k*/lnf(N) and give a randomized polynomial time algorithm to find (1 − o(1))k*/lnf(N) disjoint bases, assuming an oracle for f. We also derandomize the algorithm using minwise independent permutations and give a deterministic algorithm that finds (1 − ε)k*/lnf(N) disjoint bases. The bound we obtain is almost tight because it is known there are polymatroids for which m* < (1 + o(1))k*/lnf(N). Moreover it is known that unless NP ⊆ DTIME(nlog log n), for any ε > 0, there is no polynomial time algorithm to obtain a (1 + ε)/lnf(N)‐approximation to m*. Our result generalizes and unifies two results in the literature. © 2009 Wiley Periodicals, Inc. Random Struct. Alg., 2009

Low discrepancy sets yield approximate min-wise independent permutation families

Motivated by a problem of filtering near-duplicate Web documents, Broder, Charikar, Frieze and Mitzenmacher defined the following notion of ϵ-approximate min-wise independent permutation families . A multiset F of permutations of {0,1,…,n−1} is such a family if for all K⫅{0,1,…,n−1} and any x∈K, a permutation π chosen uniformly at random from F satisfies | Pr[ min{π(K)}=π(x)]− 1 |K| |≤ ϵ |K| . We show connections of such families with low discrepancy sets for geometric rectangles, and give explicit constructions of such families F of size n O( logn ) for ϵ=1/n Θ(1) , improving upon the previously best-known bound of Indyk. We also present polynomial-size constructions when the min-wise condition is required only for |K|≤2 O( log 2/3n) , with ϵ≥2 − O( log 2/3n) .

Completeness and robustness properties of min-wise independent permutations

We provide several new results related to the concept of min-wise independence. Our main result is that any randomized sampling scheme for the relative intersection of sets based on testing equality of samples yields an equivalent min-wise independent family. Thus, in a certain sense, min-wise independent families are “complete” for this type of estimation. We also discuss the notion of robustness, a concept extending min-wise independence to allow more efficient use of it in practice. A surprising result arising from our consideration of robustness is that under a random permutation from a min-wise independent family, any element of a fixed set has an equal chance to get any rank in the image of the set, not only the minimum as required by definition. © 2001 John Wiley & Sons, Inc. Random Struct. Alg., 18: 18–30, 2001