Indexing domains of instability for Hamiltonian systems

Abstract

Based upon the understanding of the global topologies of the singular subset, its complement, and the hyperbolic subset in the symplectic group, in this paper we study the domains of instability for hyperbolic Hamiltonian systems and define a characteristic index for such domains. This index is defined via the Maslov-type index theory for symplectic paths starting from the identity defined by C. Conley, E. Zehnder, and Y. Long, and the hyperbolic index of symplectic matrices. The old problem of the relation between the non-degenerate local minimality and the instability of hyperbolic extremal loops in the calculus of variation is also studied via this new index for the domains of instability.