We have studied numerically the dynamics of spin glasses with Ising and XY symmetry (gauge glass) in space dimensions 2, 3, and 4. The nonequilibrium spin-glass susceptibility and the nonequilibrium energy per spin of samples of large size L_b are measured as a function of anneal time t_w after a quench to temperatures T. The two observables are compared to the equilibrium spin-glass susceptibility and the equilibrium energy, respectively, measured as functions of temperature T and system size L for a range of system sizes. For any time and temperature a nonequilibrium time-dependent length scale L*(t_w,T) can be defined by comparing equilibrium and nonequilibrium quantities. Our analysis shows that for all systems studied, an "effective dynamical critical exponent" parametrization L*(t_w,T) = A(T) t^(1/z(T)) fits the data well at each temperature within the whole temperature range studied, which extends from well above the critical temperature to near T = 0 for dimension 2, or to well below the critical temperature for the other space dimensions studied. In addition, the data suggest that the dynamical critical exponent z varies smoothly when crossing the transition temperature.