The aim of this study was to test brain tumor interactions with brain networks, thereby identifying protective features and risk factors for memory recovery after resection. Seventeen patients with diffuse nonenhancing glioma (ages 22-56 years) underwent longitudinal MRI before and after surgery, and during a 12-month recovery period (47 MRI scans in total after exclusion). After each scanning session, a battery of memory tests was performed using a tablet-based screening tool, including free verbal memory, overall verbal memory, episodic memory, orientation, forward digit span, and backward digit span. Using structural MRI and neurite orientation dispersion and density imaging (NODDI) derived from diffusion-weighted images, the authors estimated lesion overlap and neurite density, respectively, with brain networks derived from normative data in healthy participants (somatomotor, dorsal attention, ventral attention, frontoparietal, and default mode network [DMN]). Linear mixed-effect models (LMMs) that regressed out the effect of age, gender, tumor grade, type of treatment, total lesion volume, and total neurite density were used to test the potential longitudinal associations between imaging markers and memory recovery. Memory recovery was not significantly associated with either the tumor location based on traditional lobe classification or the type of treatment received by patients (i.e., surgery alone or surgery with adjuvant chemoradiotherapy). Nonlocal effects of tumors were evident on neurite density, which was reduced not only within the tumor but also beyond the tumor boundary. In contrast, high preoperative neurite density outside the tumor but within the DMN was associated with better memory recovery (LMM, p value after false discovery rate correction [Pfdr] < 10-3). Furthermore, postoperative and follow-up neurite density within the DMN and frontoparietal network were also associated with memory recovery (LMM, Pfdr = 0.014 and Pfdr = 0.001, respectively). Preoperative tumor and postoperative lesion overlap with the DMN showed a significant negative association with memory recovery (LMM, Pfdr = 0.002 and Pfdr < 10-4, respectively). Imaging biomarkers of cognitive recovery and decline can be identified using NODDI and resting-state networks. Brain tumors and their corresponding treatment affecting brain networks that are fundamental for memory functioning such as the DMN can have a major impact on patients' memory recovery.