This retrospective study sought to identify patient-, provider-, and system-level characteristics associated with anxiety diagnostic specificity in mental health clinics in the Veterans Health Administration. It used administrative data extracted from Veteran Health Administration outpatient records to identify patients with a new anxiety or trauma-related diagnosis in fiscal year 2019 (N = 383,418). Logistic regression was used to model the probability of receiving an unspecified anxiety diagnosis as a function of patient-, provider-, and system-level characteristics. Unspecified anxiety disorder was diagnosed in 27% of the sample, with higher rates in primary care mental health integration clinics (42.6%) than in general mental health clinics (22.4%). Patient demographic and clinical characteristics including gender and comorbid diagnoses; provider type; provider use of an anxiety screening instrument; and facility size, complexity, and location were significant predictors of unspecified anxiety diagnosis. Anxiety diagnosis is a complex process influenced by multiple patient-, provider-, and system-level characteristics. Additional assessment tools and guidance for differential diagnosis are needed to support mental health providers in busy clinical settings to facilitate accurate and timely diagnosis of anxiety disorders.