The invariance principle is one important design principle in low-level pattern recognition. In the first part of this paper we investigate some theoretical aspects of this principle. In the second part we show how an adaptive filter system reacts to pattern sets that involve some kind of invariance. In its most general, group-theoretical form the invariance principle can be formulated as follows: We assume that we have a class of patterns of interest { p i : i ϵ I} and a given pattern q. The problem is to decide if q was a pattern of interest or not, i.e., if q = p i 0 for some i 0 ϵ I. If the patterns p i are generated from a common prototype pattern p 0 with the help of transformations g i and if these transformations g i form a group G then this problem can be solved by finding maximum correlation filter sets. The relation between these filter sets and the prototype pattern p 0 and the transformation group G is completely known. The construction of the filter functions is, however, based on the assumption that all transformations g i ϵ G (and therefore also all patterns p i ) appear equally often. The purpose of this paper is the generalization of the filter construction process to pattern classes whose elements obey some probability distribution. In the special cases of 2-D rotation invariance and scale invariance, we will show how the maximum correlation filters depend on the prototype pattern, the transformation group, and the probability distribution. We will also sketch the theory for the case of arbitrary compact transformation groups and what type of problems turn up in this case. In the last part of the paper we describe an adaptive filter system that can find the maximum correlation filter functions automatically from a given set of patterns of interest. This filter system also uses a kind of maximum separation principle so that the resulting feature vectors can be easily used by classifiers. We demonstrate how such a filter system stabilizes in states that correspond to the theoretically derived filter functions and how this filter system reacts to invariance properties in the trainings set.