Dengue is currently the fastest-spreading mosquito-borne viral illness in the world, with over half of the world's population living in areas at risk of dengue. As dengue continues to spread and become more of a health burden, it is essential to have tools that can predict when and where outbreaks might occur to better prepare vector control operations and communities' responses. One such predictive tool, the Early Warning and Response System for climate-sensitive diseases (EWARS-csd), primarily uses climatic data to alert health systems of outbreaks weeks before they occur. EWARS-csd uses the robust Distribution Lag Non-linear Model in combination with the INLA Bayesian regression framework to predict outbreaks, utilizing historical data. This study seeks to validate the tool's performance in two states of Colombia, evaluating how well the tool performed in 11 municipalities of varying dengue endemicity levels. The validation study used retrospective data with alarm indicators (mean temperature and rain sum) and an outbreak indicator (weekly hospitalizations) from 11 municipalities spanning two states in Colombia from 2015 to 2020. Calibrations of different variables were performed to find the optimal sensitivity and positive predictive value for each municipality. The study demonstrated that the tool produced overall reliable early outbreak alarms. The median of the most optimal calibration for each municipality was very high: sensitivity (97%), specificity (94%), positive predictive value (75%), and negative predictive value (99%; 95% CI). The tool worked well across all population sizes and all endemicity levels but had slightly poorer results in the highly endemic municipality at predicting non-outbreak weeks. Migration and/or socioeconomic status are factors that might impact predictive performance and should be further evaluated. Overall EWARS-csd performed very well, providing evidence that it should continue to be implemented in Colombia and other countries for outbreak prediction.