This study investigates the applicability and relevance of the residence time to the characterisation of airborne pollutant removal in the urban environment. Using large-eddy simulations of flow over an idealised building array and a realistic urban area, the statistical and spatial distributions of residence times are analysed for different source locations and wind conditions. For the idealised building array, the mean residence times in the avenue and intersection units depend on the mean flow, turbulent kinetic energy and wind direction. The probability density function (PDF) shows distinct early and late regimes when there are pollutant removal processes associated with different boundaries. For the realistic urban domain, the residence times are significantly shorter and stronger fluctuations are observed in the PDF, especially when the inflow winds are driven by the output from a high-resolution mesoscale model. The residence time is especially useful for applications to assessment and planning, when information about the timescale of pollutant removal is required and the flow is inhomogeneous and unsteady.