The loss of situation awareness (SA) caused by the out-of-the-loop phenomenon is a major problem for successful take-overs after highly automated driving. Despite recent advances in driving automation, a human backup for the automated system is currently indispensable. For uneventful situations, the automation works and drivers can perform engaging non-driving-related tasks (NDRTs). However, they need to be ready to respond at all times in case of a take-over request (TOR). For smooth and safe transition from automated to manual control, SA has to be re-established within seconds. We investigated whether regular out-the-window looks can facilitate this process. This driving simulation study is the first to investigate whether such situation awareness maintenance can improve take-over performance after a TOR. In a repeated-measures design, 89 participants experienced 5-minutes of highly automated driving with a NDRT followed by a critical take-over situation. Every 30 s, participants were asked either to monitor the traffic scene for 3 s (monitoring condition) or to merely interrupt the NDRT for 3 s (control condition). We ensured that incidental traffic monitoring was impossible during the NDRT. In the control condition, participants saw a blank screen during the 3-seconds intervals. Contrary to common intuition, the results show neither improvements in objective take-over performance nor in perceived SA, driver availability, criticality, stress, risk assessment, or adequacy of the TOR. We conclude that brief interruptions of the NDRT to look at the ongoing traffic are unable to maintain the driver's SA during highly automated driving. Future studies should therefore focus on other measures to optimize the recovery of SA.