Combinatorial rigidity of incidence systems and application to dictionary learning

Abstract

Listen

Translate article

Given a hypergraph H with m hyperedges and a set Q of mpinning subspaces, i.e. globally fixed subspaces in Euclidean space Rd, a pinned subspace-incidence system is the pair (H,Q), with the constraint that each pinning subspace in Q is contained in the subspace spanned by the point realizations in Rd of vertices of the corresponding hyperedge of H. For a subclass of pinned subspace-incidence systems where all pinning subspaces are of dimension 1, this paper provides a combinatorial characterization of minimal rigidity, i.e. those systems that are guaranteed to generically yield a locally unique realization.Pinned subspace-incidence systems arise as a geometric interpretation of the Dictionary Learning (aka sparse coding) problem, i.e. the problem of obtaining a sparse representation of a given set of data vectors by learning dictionary vectors upon which the data vectors can be written as sparse linear combinations. Viewing the dictionary vectors from a geometry perspective as the spanning set of a subspace arrangement, we provide a systematic classification of problems related to dictionary learning together with various algorithms, their assumptions and performance. We formally prove the intuitively expected bound that the size of dictionary cannot be significantly less than the number of data vectors when the data are generic or uniformly distributed, and gives a way of constructing a dictionary that meets the bound. For less stringent restrictions on data, but a natural modification of the dictionary learning problem, we provide a further dictionary learning algorithm by leveraging the well-known DR-planning technique from geometric constraint solving. Although there are recent rigidity based approaches for low rank matrix completion, we are unaware of prior application of combinatorial rigidity techniques in the setting of Dictionary Learning.Other applications of pinned subspace-incidence systems include modeling microfibrils in biomaterials such as cellulose and collagen.