Neoadjuvant chemotherapy followed by surgery is Japan's most effective treatment modality for advanced thoracic esophageal squamous cell carcinoma. However, the prognosis is not as expected. This study aimed to examine prognostic factors in patients with pathologically ineffective neoadjuvant chemotherapy followed by surgery for esophageal squamous cell carcinoma. We retrospectively analyzed patients who underwent neoadjuvant chemotherapy followed by curative esophagectomy for esophageal squamous cell carcinoma between December 2008 and July 2021. The patients were divided into the neoadjuvant chemotherapy effective group and the neoadjuvant chemotherapy ineffective group according to the pathological diagnosis. Clinicopathological data, prognosis, and recurrence were analyzed. A total of 143 patients (121 males, 22 females; median age, 67years) were included in this study. Of these, 34 patients were classified into the effective group and the remaining 109 patients were assigned to the ineffective group. The ineffective group had significantly worse overall survival and recurrence-free survival than the effective group (p = 0.0192 and p = 0.0070, respectively). In the ineffective group, multivariate analysis demonstrated that microscopic venous invasion was an independent prognostic factor for overall survival (hazard ratio 2.44; 95% confidence interval 1.13-5.30) and recurrence-free survival (hazard ratio 2.43; 95% confidence interval 1.24-4.73). Microscopic venous invasion was associated with poor survival and cancer recurrence in the neoadjuvant chemotherapy ineffective group of patients who underwent esophagectomy for esophageal squamous cell carcinoma.