Background: Postoperative Atrial Fibrillation (POAF) is a common complication in cardiac surgery. Despite its multifactorial origin, the left atrial (LA) size is closely linked to POAF, raising the question of a valid cut-off value and its impact on the long-term outcome. Methods: Patients without a history of AF who underwent coronary artery bypass grafting between 2014 and 2016 were selected for this retrospective study. LA size was preoperatively assessed using the left atrial anterior-posterior diameter (LAAPd). Correlation and logistic regression analyses were performed, following a receiver-operating characteristic (ROC) analysis. Propensity score matching (PSM) was applied to ensure group comparability, followed by a comparison analysis regarding the primary endpoint of POAF and the secondary endpoints of all-cause mortality and stroke during a five-year follow-up. Results: A total of 933 patients were enrolled in the study eventually revealing a significant correlation between LAAPd and POAF (cor = 0.09, p < 0.01). A cut-off point of 38.5 mm was identified, resulting in groups with 366 patients each after PSM. Overall, patients with a dilated LA presented a significantly higher rate of POAF (22.3% vs. 30.4%, p = 0.02). In a five-year follow-up, a slightly higher rate of all-cause mortality (9.8% vs. 13.7%, HR 1.4 [0.92-2.29], p = 0.10) was observed, but there was no difference in the occurrence of strokes (3.6% vs. 3.3%, p = 0.87). Conclusions: An LAAPd of >38.5 mm was found to be an independent predictor of POAF after coronary artery bypass grafting and resulted in a non-significant tendency towards a worse outcome regarding all-cause mortality in a five-year follow-up.