We report on the time-resolved detection of the anomalous velocity, constituting charge carriers moving perpendicular to an electric driving field, in undoped GaAs quantum wells. For this we optically excite the quantum wells with circularly polarized femtosecond laser pulses, thereby creating a state which breaks time-inversion symmetry. We then employ a quasi-single-cycle terahertz pulse as an electric driving field to induce the anomalous velocity. The electromagnetic radiation emitted from the anomalous velocity is studied with a subpicosecond time resolution and reveals intriguing results. We are able to distinguish between intrinsic (linked to the Berry curvature) and extrinsic (linked to scattering) contributions to the anomalous velocity both originating from the valence band and observe local energy space dependence of the anomalous velocity. Our results thus constitute a significant step towards noninvasive probing of the anomalous velocity locally in the full energy-momentum space and enable the investigation of many popular physical effects such as the anomalous Hall effect and spin Hall effect on ultrafast time scales.