We study the dispersive properties of photonic crystal fibers (PCF's) with elliptical air holes. The unusual guidance of PCF leads to novel behavior of the birefringence, group-velocity walk-off, and dispersion parameters, including the possibility of zero walk-off with high birefringence in the single-mode regime. A number of these effects are closely tied to the underlying radiation states of the air-hole lattice.