Abstract Black hole X-ray binaries in their hard and hard-intermediate states display hard and soft time lags between broadband noise variations (high-energy emission lagging low-energy and vice versa), which could be used to constrain the geometry of the disk and Comptonising corona in these systems. Comptonisation and reverberation lag models, which are based on light-travel delays, can imply coronae which are very large (hundreds to thousands of gravitational radii, Rg) and in conflict with constraints from X-ray spectral modelling and polarimetry. Here we show that the observed large and complex X-ray time lags can be explained by a model where fluctuations are generated in and propagate through the blackbody-emitting disk to a relatively compact (∼10 Rg) inner corona. The model naturally explains why the disk variations lead coronal variations with a Fourier-frequency dependent lag at frequencies <1 Hz, since longer variability time-scales originate from larger disk radii. The propagating fluctuations also modulate successively the coronal seed photons from the disk, heating of the corona via viscous dissipation and the resulting reverberation signal. The interplay of these different effects leads to the observed complex pattern of lag behaviour between disk and power-law emission and different power-law energy bands, the energy-dependence of power-spectral shape, and a strong dependence of spectral-timing properties on coronal geometry. The observed spectral-timing complexity is thus a natural consequence of the response of the disk-corona system to mass-accretion fluctuations propagating through the disk.