Studies have reported higher in-hospital mortality rates in patients living with dementia (PlwD) with limited evidence across age groups, clinical departments, and admission diagnoses. The aim of this study was to compare the in-hospital mortality rate of PlwD with patients without dementia across groups, clinical departments, and admission diagnoses. This case-control study included patients aged ≥ 60 years hospitalized in 1 of 14 German hospitals between January 2019 and July 2023. PlwD were matched to patients without dementia. The associations between dementia and in-hospital mortality across groups were assessed using univariable logistic regression analyses. 15,956 patients with and 15,956 without dementia were included (mean age: 83.9 years, 60.7% female). PlwD had a significantly higher in-hospital mortality rate (14.0% vs. 11.7%; OR 1.24, 95% CI: 1.16-1.32) than non-dementia controls. The highest excess mortality rate was observed in the youngest age group (60-70 years: 10.9% vs. 5.7%; OR: 2.05, 95% CI: 1.30-3.24), decreased with age, and became non-significant in the oldest age group (≥90 years: 16.2% vs. 17.3%; OR: 0.93, 95% CI: 0.80-1.08). Significant differences were found for digestive system disorders (OR: 1.59; 95% CI: 1.15-1.89), cardiovascular and cerebrovascular disorders (OR: 1.51; 95% CI: 1.30-1.75), endocrine, nutritional, and metabolic diseases (OR: 1.42; 95% CI: 1.06-1.90), and pneumonia (OR: 1.20; 95% CI: 1.04-1.37), as well as for all clinic departments except for geriatric departments. The excess mortality rate was highest in younger age groups, where the general mortality and complication rate is relatively low in the general population. Appropriate approaches are needed, especially in non-geriatric wards.