Initial experiments which have explored the physics of the underdense (blowout) regime of the plasma wakefield accelerator (PWFA) at the Argonne Wakefield Accelerator facility are reported. In this regime, the relativistic electron beam is denser than the plasma, causing the beam channel to completely rarefy, and leaving a high quality accelerating region which also contains a uniform ion column. This ion column in turn allows the drive and accelerating beams to be well guided over many initial beam beta-function lengths. The results of these experiments, which have taken place over several years, are reviewed. Notable achievements in the course of these studies include the creation and measurement of drive and witness beam generated in an rf photoinjector, as well as previously published studies on drive beam guiding in the underdense regime. In addition, these experiments allowed measurement of both beam energy loss and gain, at a maximum average rate of 25 MeV/m in this regime of the PWFA, which is consistent with a peak acceleration gradient of 62 MeV/m in the excited waves. Difficulties associated with this type of experiment are discussed, as are prospects for mitigating these difficulties and achieving high gradient acceleration in planned future experiments.