The confinement and stability properties of the edge of a magnetically confined fusion plasma have long resisted theoretical explanation. Numerical simulation with the M3D extended MHD code shows that large periodic Edge Localized Modes (ELMs), associated with otherwise good confinement, represent a new type of nonlinear MHD plasma instability, in which a coherent plasma instability couples to the favorable part of a chaotic magnetic field. The magnetic surface bounding the plasma typically contains at least one X-point. Under small perturbations, such a Hamiltonian surface splits into two different asymptotic limits. A ballooning-type plasma instability can couple to the "unstable" field manifold that forms bulges around the original magnetic surface and then grow to penetrate deep into the plasma. The magnetic field becomes chaotic over the affected region, allowing plasma loss from the core to the outside. Eventually the instability saturates and the plasma relaxes back towards its original axisymmetric shape. The interaction between the plasma and the homoclinic field perturbation drives a multi-stage process that reproduces many experimental observations. This type of instability may help to explain the wide range of ELM and ELM-free behavior observed in fusion plasmas with X-points, as well as properties of plasma confinement.