Introduction: Memory impairment and mood disorders are among the most troubling sequelae following traumatic brain injury (TBI). The relationships between comorbid psychiatric disorders and memory function have not been well illustrated. The aim of the study was to explore the relationships of comorbid anxiety and depressive symptoms with memory function following TBI. Methods: A total of 46 TBI participants across all levels of injury and 23 healthy controls were enrolled in this case-control study. Wechsler Memory Scale-Chinese Revision (WMS-CR) picture, recognition, associative learning, comprehension memory, and digit span were administered to evaluate several categories of memory capacity. The Hospital Anxiety and Depression Scale (HADS) was employed to evaluate the anxiety and depressive symptoms. Stepwise multiple linear regressions were conducted. Results: Compared to healthy controls, the participants with TBI reported more anxiety and depressive symptoms. In the meanwhile, they performed more poorly on memory tests, showing 1.84 SDs, 1.07 SDs, and 0.68 SDs below healthy participants on visuospatial memory, working memory, and verbal memory, respectively. A variety of variables, including HADS depression, HADS anxiety, age, GCS, and education were associated with posttraumatic memory function in the bivariate models. The stepwise multiple linear regressions demonstrated a negative association between HADS depression and posttraumatic memory function, especially performance on visuospatial and verbal memory and a positive association between education and posttraumatic memory function. Conclusion: More depressive symptoms rather than anxiety symptoms and less years of education are significant predictors for posttraumatic memory dysfunction.