Background Anion gap (AG) has been proved to be associated with prognosis of many cardiovascular diseases. This study is aimed at exploring the association of AG with inhospital all-cause mortality and adverse clinical outcomes in coronary care unit (CCU) patients. Method All data of this study was extracted from Medical Information Mart for Intensive Care III (MIMIC-III, version 1.4) database. All patients were divided into four groups according to AG quartiles. Primary outcome was inhospital all-cause mortality. Lowess smoothing curve was drawn to describe the overall trend of inhospital mortality. Binary logistic regression analysis was performed to determine the independent effect of AG on inhospital mortality. Result A total of 3593 patients were enrolled in this study. In unadjusted model, as AG quartiles increased, inhospital mortality increased significantly, OR increased stepwise from quartile 2 (OR, 95% CI: 1.01, 0.74-1.38, P = 0.958) to quartile 4 (OR, 95% CI: 2.72, 2.08-3.55, P < 0.001). After adjusting for possible confounding variables, this association was attenuated, but still remained statistically significant (quartile 1 vs. quartile 4: OR, 95% CI: 1.02, 0.72-1.45 vs. 1.49, 1.07-2.09, P = 0.019). Moreover, CCU mortality (P < 0.001) and rate of acute kidney injury (P < 0.001) were proved to be higher in the highest AG quartiles. Length of CCU (P < 0.001) and hospital stay (P < 0.001) prolonged significantly in higher AG quartiles. Maximum sequential organ failure assessment score (SOFA) (P < 0.001) and simplified acute physiology score II (SAPSII) (P < 0.001) increased significantly as AG quartiles increased. Moderate predictive ability of AG on inhospital (AUC: 0.6291), CCU mortality (AUC: 0.6355), and acute kidney injury (AUC: 0.6096) was confirmed. The interactions were proved to be significant in hypercholesterolemia, congestive heart failure, chronic lung disease, respiratory failure, oral anticoagulants, Beta-blocks, angiotensin-converting enzyme inhibitor (ACEI)/angiotensin receptor blocker (ARB), and vasopressin treatment subgroups. Conclusion AG was an independent risk factor of inhospital all-cause mortality and was associated with adverse clinical outcomes in CCU patients.