Elliptic and non-hyperbolic closed characteristics on compact convex P-cyclic symmetric hypersurfaces in $$\mathbf{R}^{2n}$$

Abstract

Let $$\Sigma $$ be a compact convex hypersurface in $$\mathbf{R}^{2n}$$ which is P-cyclic symmetric, i.e., $$x\in \Sigma $$ implies $$Px\in \Sigma $$ with P being a $$2n\times 2n$$ symplectic orthogonal matrix and $$P^k=I_{2n}$$, where $$n, k\ge 2$$, $$ker(P-I_{2n})=0$$. In this paper, we first generalize Ekeland index theory for periodic solutions of convex Hamiltonian system to a index theory with P boundary value condition and study its relationship with Maslov P-index theory, then we use index theory to prove the existence of elliptic and non-hyperbolic closed characteristics on compact convex P-cyclic symmetric hypersurfaces in $$\mathbf{R}^{2n}$$ for a broad class of symplectic orthogonal matrix P.