Long gamma-ray bursts (GRBs) have been linked to extreme core-collapse supernovae from massive stars. Gravitational waves (GW) offer a probe of the physics behind long GRBs. We investigate models of long-lived (~10-1000s) GW emission associated with the accretion disk of a collapsed star or with its protoneutron star remnant. Using data from LIGO's fifth science run, and GRB triggers from the swift experiment, we perform a search for unmodeled long-lived GW transients. Finding no evidence of GW emission, we place 90% confidence level upper limits on the GW fluence at Earth from long GRBs for three waveforms inspired by a model of GWs from accretion disk instabilities. These limits range from F<3.5 ergs cm^-2 to $F<1200 ergs cm^-2, depending on the GRB and on the model, allowing us to probe optimistic scenarios of GW production out to distances as far as ~33 Mpc. Advanced detectors are expected to achieve strain sensitivities 10x better than initial LIGO, potentially allowing us to probe the engines of the nearest long GRBs.