Towards a topological–geometrical theory of group equivariant non-expansive operators for data analysis and machine learning

Abstract

We provide a general mathematical framework for group and set equivariance in machine learning. We define group equivariant non-expansive operators (GENEOs) as maps between function spaces associated with groups of transformations. We study the topological and metric properties of the space of GENEOs to evaluate their approximating power and set the basis for general strategies to initialize and compose operators. We define suitable pseudo-metrics for the function spaces, the equivariance groups and the set of non-expansive operators. We prove that, under suitable assumptions, the space of GENEOs is compact and convex. These results provide fundamental guarantees in a machine learning perspective. By considering isometry-equivariant non-expansive operators, we describe a simple strategy to select and sample operators. Thereafter, we show how selected and sampled operators can be used both to perform classical metric learning and to inject knowledge in artificial neural networks. Controlling the flow and representation of information in deep neural networks is fundamental to making networks intelligible. Bergomi et al introduce a mathematical framework in which the space of possible operators representing the data is constrained by using symmetries. This constrained space is still suitable for machine learning: operators can be efficiently computed, approximated and parameterized for optimization.