Low health literacy has been associated with adverse outcomes in health maintenance and the course of chronic physical illness. In particular, anxiety disorders can also affect one's physical health, causing issues including cardiovascular, respiratory, gastrointestinal and immune system disorders. However, there are no reports on physical health literacy among Japanese patients with mental illness. A patient background questionnaire, the Japanese version of the Ten-Item Personality Inventory, and the Japanese version of the Health Literacy Scale (HLS-EU-Q47; European Health Literacy Survey Questionnaire) were distributed face to face to 1000 psychiatric outpatients. A total of 785 valid responses including 211 patients with schizophrenia, 261 patients with mood disorders, and 234 patients with anxiety disorders were obtained by mail. Health literacy was "limited" in 52% of patients with schizophrenia, 51% of those with mood disorders, and 38% of those with anxiety disorders. Among patients with mood disorders, there were no differences between those with major depressive disorder and those with bipolar disorder. Anxiety disorders were associated with higher health literacy than schizophrenia and mood disorders (odds ratio (OR) 1.85, 95% confidence interval (95% CI) 1.07, 3.34), and in terms of personality, neuroticism (OR 0.85, 95% CI 0.75, 0.97) and openness (OR 0.85, 95% CI 0.74, 0.98) were associated with limited health literacy, while agreeableness (OR 1.36, 95% CI 1.18, 1.57) and extraversion OR 1.34, 95% CI 1.17, 1.52) were associated with higher health literacy. The results of this study indicate limited health literacy in patients with mental illness, in particular, limited health literacy in outpatients with schizophrenia and mood disorders. In addition, gender and some personality traits were associated with physical health literacy. Based on these results, physical health education should be individualized.