Outlying observations can bias regression estimates, requiring the use of robust estimators. Comparing robust estimates to those obtained using OLS is a common robustness check, however, such comparisons have been mostly informal due to the lack of available tests. Here we introduce a formal test for coefficient distortion due to outliers in regression models. Our proposed test is based on the difference between OLS and robust estimates obtained using a class of Huber-skip M-type estimators (such as Impulse Indicator Saturation or Robustified Least Squares). Establishing asymptotics of the corresponding Huber-skip M-estimators using an empirical process CLT recently developed by Berenguer-Rico et al. (2019), we show that our distortion test has an asymptotic chi-squared distribution, and is valid for cross-sectional as well as panel and time series models. To improve finite sample performance and to alleviate concerns on distributional assumptions, we further introduce and explore three bootstrap testing schemes. We apply our outlier distortion test to estimates of the macro-economic impacts of climate change allowing for adaptation. We find that OLS estimates are significantly different to those obtained using a robust estimator and provide evidence of income-driven adaptation. Projecting the resulting damage curve to the end of the century shows that outlier-robust estimates dampen projected GDP losses and reduce the estimated marginal impacts of additional warming under adaptation.