After the financial crisis, evaluating the financial health of banks under stressed scenarios has become common practice among supervisors. According to supervisory guidelines, the adverse scenarios prepared for stress tests need to be severe but plausible. The first contribution of this paper is to propose a model-based approach to assess the severity of the scenarios. To do so, we use a Large Bayesian VAR model estimated on the Italian economy where potential spillovers between the macroeconomy and the aggregate banking sector are explicitly considered. We show that the 2018 exercise has been the most severe so far, in particular, due to the path of GDP, the stock market index and the 3-month Euribor rate. Our second contribution is an evaluation of whether the resilience of the Italian banking sector to adverse scenarios has increased over time (for example, thanks to improved risk management practices induced by greater awareness of risks that come with performing stress test exercises). To this scope, we construct counterfactual exercises by recalibrating the scenarios of the 2016 and 2018 exercises so that they have the same severity as the 2014 exercise. We find that in 2018, the economy would have experienced a smaller decline in loans compared to the previous exercises. This implies that banks could afford to deleverage less, i.e. maintain a higher exposure to risk in their balance sheets. We interpret this as evidence of increased resilience.