We consider the motion of a single tracer particle on a three-dimensional lattice in the presence of hard, immobile obstacles at low density. Starting from an equilibrium state, a constant pulling force on the tracer particle is switched on. We elaborate a complete solution for the dynamics, exact in first order of the obstacle density. The time-dependent velocity response and the fluctuations along and perpendicular to the force are derived and compared to stochastic simulations. The non-analytic dependence of the linear-response functions on the frequency, reflecting the long-time tail of the velocity autocorrelation function in equilibrium, has a non-equilibrium analogue in the non-analytic dependence of the stationary velocity as well as the diffusion coefficients as a function of the force. We show that a divergent time scale emerges, separating a regime where linear response is qualitatively correct from a regime where nonlinear effects dominate.