Nonconvexity of the Generalized Numerical Range Associated with the Principal Character

Abstract

AbstractSuppose m and n are integers such that 1 ≤ m ≤ n. For a subgroup H of the symmetric group Sm of degree m, consider the generalized matrix function on m × m matrices B = (bij) defined by and the generalized numerical range of an n × n complex matrix A associated with dH defined byIt is known that WH(A) is convex if m = 1 or if m = n = 2. We show that there exist normal matrices A for which WH(A) is not convex if 3 ≤ m ≤ n. Moreover, for m = 2 < n, we prove that a normal matrix A with eigenvalues lying on a straight line has convex WH(A) if and only if νA is Hermitian for some nonzero ν ∈ ℂ. These results extend those of Hu, Hurley and Tam, who studied the special case when 2 ≤ m ≤ 3 ≤ n and H = Sm.