The Levitated Dipole Experiment (LDX) explores confinement and stability of plasma created within the dipole field of a strong superconducting magnet. During initial experiments, long-pulse, quasi-steady state discharges that last more than 10 s and have peak beta of more than 20% are studied. The plasma is created by multi-frequency electron cyclotron resonance heating (ECRH) at 2.45 and 6.4 GHz. A population of energetic electrons, with mean energies above 50 keV, dominates the plasma pressure. Creation of high pressure, high beta plasma is possible only when intense hot electron interchange (HEI) instabilities are stabilized by sufficient neutral gas fueling. The instabilities resonate with the magnetic drift motion of the energetic electrons and can cause rapid radial transport. Measurements of the electrostatic and magnetic fluctuations of the HEI instability are described along with observations of the instability’s spectral characteristics. Fluctuations of the outer poloidal field induced by the HEI show a rapid evolution of the perturbed pressure profile.