Earlier studies demonstrated that visual tracking of dynamic objects is supported by both scene-based and object-based reference frames, depending on the magnitude of scene displacement (Huff, Jahn, & Schwan, 2009; Liu et al., 2005). The current experiment tests if this pattern also applies to younger participants, i.e. school-age children, by comparing the effects of abrupt scene rotations on tracking performance of multiple dynamic objects in a 3D scene across five age groups (grade 1, 3, 5, 7 and adults). Scene rotations have two consequences: displacement of (1) the whole scene and, (2) individual objects. Tracking accuracy of 123 participants was measured across five age groups (grades 1, 3, 5, 7, and adults). Either 1 or 3 targets moved independently among a total of 8 identical objects for 5s. The scene remained constant or was rotated by 10° or 20° after 3s. Tracking performance of all participants was well above chance level (probability of 0.5) and an age-related increase in performance was observed. Contrasting the two factors revealed that scene rotation had a greater impact on performance than object displacement. Further, the effect of abrupt rotations was independent of age. These findings suggest that allocentric reference frames support attentive tracking across abrupt viewpoint changes and that scene-based tracking is already applied early in human development. Findings are discussed in light of new studies that link MOT to grouping processes (local and global). We propose that scene-based or allocentric processing abilities undergo a similar development as, or are connected to, grouping skills.