This research introduces an innovative approach to achieve a computationally efficient model for calculating impulse responses in an urban environment, more precisely a street canyon, for auralization purposes. The image source method has shown its merits before, but when used for auralization, it typically lacks realism. In this study the image source method is used to create a basic impulse response of a street canyon, enriched by a stochastic response based on an envelope function. While the image source model accounts for the most prominent reflections, the stochastic part is due to all scattered sound energy in the street originating mainly from facade irregularities and street furniture. To obtain realistic frequency dependent envelope functions for various street canyon geometries, the function is based on measured impulse responses in nearly 100 inner-city street canyons in the city of Ghent, which were conducted by Thomas et al. (2013). By the proposed approach, we are able to generate impulse responses for street canyons suitable for sound perception research in an efficient way. More studies are needed to verify the applicability of the model for streets outside the Ghent dataset.