To identify factors associated with grade ≥3 treatment related late esophageal toxicity after lung or liver stereotactic body radiation therapy (SBRT). This was a retrospective review of 52 patients with a planning target volume within 2 cm of the esophagus from a prospective registry of 607 lung and liver SBRT patients treated between 2005 and 2011. Patients were treated using a risk-adapted dose regimen to a median dose of 50 Gy in 5 fractions (range, 37.5-60 Gy in 3-10 fractions). Normal structures were contoured using Radiation Therapy Oncology Group (RTOG) defined criteria. The median esophageal point dose and 1-cc dose were 32.3 Gy (range, 8.9-55.4 Gy) and 24.0 Gy (range, 7.8-50.9 Gy), respectively. Two patients had an esophageal fistula at a median of 8.4 months after SBRT, with maximum esophageal point doses of 51.5 and 52 Gy, and 1-cc doses of 48.1 and 50 Gy, respectively. These point and 1-cc doses were exceeded by 9 and 2 patients, respectively, without a fistula. The risk of a fistula for point doses exceeding 40, 45, and 50 Gy was 9.5% (n=2/21), 10.5% (n=2/19), and 12.5% (n=2/16), respectively. The risk of fistula for 1-cc doses exceeding 40, 45, and 50 Gy was 25% (n=2/9), 50% (n=2/4), and 50% (n=2/4), respectively. Eighteen patients received systemic therapy after SBRT (11 systemic chemotherapy, and 6 biologic agents, and 1 both). Both patients with fistulas had received adjuvant anti-angiogenic (vascular endothelial growth factor) agents within 2 months of completing SBRT. No patient had a fistula in the absence of adjuvant VEGF-modulating agents. Esophageal fistula is a rare complication of SBRT. In this series, fistula was seen with esophageal point doses exceeding 51 Gy and 1-cc doses greater than 48 Gy. Notably, however, fistula was seen only in those patients who also received adjuvant VEGF-modulating agents after SBRT. The potential interaction of dose and adjuvant therapy should be considered when delivering SBRT near the esophagus.