Background: Primary thyroid lymphoma (PTL) is known to develop mostly in patients with Hashimoto's thyroiditis (HT), and it is rare for it to develop in patients with Graves' disease (GD). The objective of this study was to investigate the clinical features, pathological findings, and long-term outcomes of PTL patients, grouped according to the presence of GD, HT, or no autoimmune thyroid disease (AITDs). The GD group was of major interest due to limited knowledge of the relationship with PTL. Methods: In this single-center retrospective cohort study, we reviewed the medical records of all patients diagnosed with PTL between August 1979 and October 2021, and we characterized the patients according to the presence of HT, GD, or no AITDs. Pathological specimens were classified according to the World Health Organization classification. Staging was performed in accordance with the Ann Arbor classification. Results: During the 42-year period, 498 participants were diagnosed with PTL. The median age was 68 (interquartile range 61-76) years, and 221 patients were stage IE, whereas the remaining 277 patients were stage IIE. Of the PTL patients, 431 (86.6%) were diagnosed with HT, 9 (1.8%) were diagnosed with GD, and 58 (11.6%) did not have AITDs. All nine patients with GD were positive for anti-thyroglobulin antibody and/or anti-thyroid peroxidase antibody. All patients with GD were treated with anti-thyroid medication. There were no significant differences in the proportions of each subtype of PTL between the PTL patients with GD and all subjects with PTL (p = 0.51), PTL patients with HT (p = 0.51), or PTL patients without AITDs (p = 0.48). The median follow-up time was 6.2 (interquartile range 3.0-10.7) years after the diagnosis of PTL. The Kaplan-Meier curve analyses showed no significant differences in overall survival and event-free survival between PTL patients with GD and those with HT (p = 0.37), or between PTL patients with GD and those without AITDs (p = 0.43). Conclusions: The PTL was observed with HT in a majority of cases, and rarely with GD (1.8%). The proportions of each pathological subtype of PTL and the prognosis of PTL were not different between the patients with GD and those with HT or those without AITDs.