Cytomegaloviruses (CMVs) extensively rearrange the cellular membrane system to develop assembly compartment (AC), but the earliest events in this process are poorly characterized. Here, we demonstrate that murine CMV (MCMV) infection restrains endosomal trafficking of cargo molecules that travel along the recycling (TfR and MHC-I) and the late endosomal (EGFR, M6PR, Lamp1) circuit. Internalized cargo accumulates in Arf6-, Rab5-, Rab22A-, and Rab11-positive and Rab35-, Rab8-, and Rab10-negative juxtanuclear endosomes, suggesting the disruption of Arf/Rab regulatory cascade at the stage of sorting endosomes and the endosomal recycling compartment. Rearrangement of the endosomal system is initiated by an MCMV-encoded function very early in the infection. Our study, thus, establishes a set of landmarks of endosomal remodeling in the early phase of MCMV-infection which coincide with the Golgi rearrangement, suggesting that these perturbations are the earliest membrane reorganizations that may represent an initial step in the biogenesis of the AC.