Many researchers have focused on the role of the autonomic nervous system in the tumor microenvironment. Autonomic nerves include the sympathetic and parasympathetic nerves, which are known to induce cancer growth and metastasis. However, in salivary duct carcinoma (SDC), a rare and highly malignant tumor, the issue should be investigated from both biological and therapeutic perspectives. We explored the clinicopathological and prognostic implications of the autonomic nerves in 129 SDCs. Immunohistochemistry was performed to determine the nature of each nerve using antibodies against S100, tyrosine hydroxylase (TH) as a sympathetic marker, and vesicular acetylcholine transporter (VAChT) as a parasympathetic marker. The area of each marker-positive nerve was digitized and evaluated quantitatively. Double immunofluorescence for TH and VAChT was performed in selected cases. The expression of the secreted neurotrophins was also examined. S100-positive nerves were present in the cancer tissue in 94 of 129 cases (72.9%). Among them, TH-positive sympathetic nerves and/or VAChT-positive parasympathetic nerves were identified in 92 cases (97.9%), and 59 cases (62.8%) had TH/VAChT-co-expressing nerves. Double immunofluorescence revealed a mosaic pattern of sympathetic and parasympathetic fibers in co-expressing nerve bundles. The presence of autonomic nerves, regardless of their area, was significantly associated with aggressive histological features, advanced T/N classification, and a poor prognosis, with shorter disease-free and overall survival. There was an association between some tumor immune microenvironment-related markers and the autonomic nerve status, but not the latter and the secreted neurotrophin expression. This study suggests that autonomic nerves might play a role in the progression of SDC.