Patients (pts) with brain metastases (BM) treated with stereotactic radiosurgery (SRS) alone are at increased risk of distant brain failure (DBF). A nomogram and recursive partitioning analysis (RPA) designed to predict for DBF at 1 year (yr) after SRS alone from a single institution Canadian tertiary care hospital was recently published (Rodrigues et al., Radio and Oncol 2014). We sought to evaluate the external validity of these tools using a multi-institutional independent validation cohort (VC). The records of consecutive pts with BM treated with SRS alone at 2 tertiary care cancer institutes between 2005-2013 were reviewed. Inclusion criteria were identical to the Canadian study and included pts treated with initial SRS alone, up to 3 BM, and no previous cranial irradiation. The nomogram variables included age, number of BM, largest lesion volume (cc), and WHO performance status (PS) to calculate probability of 1-yr DBF using the Kaplan-Meier method. The RPA defined 3 risk categories (low [LR], intermediate [IR], and high risk [HR]) based on PS, age, and number of BM. Discrimination was assessed using Harrell’s c-index and calibration was assessed using calibration plots and correlation between expected and observed DBF. After exclusions, a total of 282 pts with 390 BM and 303 pts with 416 BM were eligible for the nomogram and RPA VC, respectively. Median age was 60 years old, median tumor volume was 1.35 cc, and median number of BM was 1. Primary disease site was lung 44%, melanoma 20.2%, breast 17.4%, renal 7.8%, GI 2.8%, and other 11.6%. One-year actuarial DBF for the nomogram VC was 50.9% compared to 43.9% in the Canadian study. The Harrell’s c-index for the DBF nomogram using the VC was 0.56 compared to 0.69 in the Canadian study, indicating poor discrimination ability. Calibration plots for the nomogram demonstrated good correlation between predicted and observed DBF (r2=0.92), but with systematic underestimation of DBF risk by an average of 19.6% across all risk levels. RPA risk groups LR, IR, and HR had 1-yr DBF rates of 39%, 62%, and 51.8%, compared to Canadian rates of 25.1%, 45.9%, and 69.1%, respectively. RPA risk groups did demonstrate significantly different actuarial DBF rates (log-rank p=0.0008), but IR had a higher 1-yr DBF rate than HR and would have been categorized as high risk in the Canadian publication (> 40%). The accuracy of the RPA classification was 53% compared with 75% in the Canadian study. In an independent multi-institutional cohort, the Canadian nomogram and RPA classification were not found to be externally valid, had limited ability to accurately differentiate DBF at 1-yr, and generally underestimated risk of DBF. We currently recommend caution in using these tools to predict individualized risk of 1-yr DBF after SRS alone.