We have considered the evolution of dust orbits in a gravitational and electromagnetic field like that in the magnetosphere of Saturn. The orbits evolve due to the action of plasma drag and gyrophase drag (Lorentz force combined with a delay in charging of the dust). We calculate the average relative change in the semimajor axis, 〈 a dot /a 0〉 , and the average change in eccentricity, 〈 e dot 〉 , for different initial orbits and different plasma conditions, including cases with constant temperature and density and cases with gradients in temperature and density. Finally, we adopted a plasma model which may be appropriate for the E-ring plasma of today. This model has a cold and hot electron component and the production of secondary electrons is important, contrary to the other cases considered. It appears possible that the E-ring dust particles, if injected by Encleadus at low eccentricities, will keep their low eccentricities in today's plasma environment. This would imply that the orbits would drift slowly outward mainly due to plasma drag. Using an extensive set of different parameters, we show that in some circumstances the perturbing forces may be strongly dependent on global or local plasma conditions and/or dust properties. This implies that even moderate environmental changes could have significant effects on the evolution of Saturn's E-ring, and emphasizes the need for better in situ measurements in order to understand this diffuse ring phenomenon.