The use of outdoor virtual scenarios promises to have a lot of potential to facilitate reproducible sensory evaluation experiments in laboratory. Auralizations allow for the integration of simulated or measured sound sources and transfer paths between the sources and receivers. Nonetheless, pure simulations can lack perfect plausibility. This contribution investigates the augmentation of auralized outdoor scenes based on simulated impulse responses (IRs) by ambient or background sounds. For this purpose, foreground events such as car pass-bys are created by simplified simulation of impulse responses. Due to their large number of events, however, ambient sounds are typically not simulated. Instead, spherical microphone array recordings can be used to capture the background sound. Using synthesized car sounds, we examine how much the augmentation by background sound improves the auditory plausibility of simulated impulse responses in comparison with the equivalent measured ones.