This paper focuses on the impact of financial fragmentation on small and medium enterprises’ (SMEs) access to finance. We combine country-level data on financial fragmentation and the ECB’s SAFE (Survey on the Access to Finance of Enterprises) data for 12 European Union (EU) countries over 2009–2016. Our findings indicate that an increase in financial fragmentation not only raises the probability of all firms to be rationed but also to be charged higher loan rates; in addition, it increases the likelihood of borrower discouragement and it impairs firms’ perceptions of the future availability of bank funds. Less creditworthy firms are even more likely to become credit rationed, suggesting a flight to quality effect in lending. However, our study also documents a potential adverse effect of increasing bank market power resulting from greater integration. This suggests that financial integration could impair firms’ financing, if not accompanied by policy initiatives aimed at maintaining an optimal level of competition in the banking sector.