The structured Gerstenhaber problem (I)

Abstract

Let b be a symmetric or alternating bilinear form on a finite-dimensional vector space V. When the characteristic of the underlying field is not 2, we determine the greatest dimension for a linear subspace of nilpotent b-symmetric or b-alternating endomorphisms of V, expressing it as a function of the dimension, the rank, and the Witt index of b. Similar results are obtained for subspaces of nilpotent b-Hermitian endomorphisms when b is a Hermitian form with respect to a non-identity involution. In three situations (b-symmetric endomorphisms when b is symmetric, b-alternating endomorphisms when b is alternating, and b-Hermitian endomorphisms when b is Hermitian and the underlying field has more than 2 elements), we also characterize the linear subspaces with the maximal dimension.Our results are wide generalizations of results of Meshulam and Radwan [7], who tackled the case of a non-degenerate symmetric bilinear form over the field of complex numbers, and recent results of Kokol Bukovšek and Omladič [4], in which the spaces with maximal dimension were determined when the underlying field is the one of complex numbers, the bilinear form b is symmetric and non-degenerate, and one considers b-symmetric endomorphisms.